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EXECUTIVE  SUMMARY 


INTRODUCTION 

In  February  2003,  the  Naval  Research  Laboratory  (NRL)  Airborne  Multi-sensor 
Towed  Array  Detection  System  (aMTADS)  and  the  Oak  Ridge  National  Laboratory 
(ORNL)  Airborne  Geophysics  System  (ORAGS)  magnetometer  systems  surveyed  600 
hectares  of  a  fonner  practice  bombing  range  at  Isleta  Pueblo,  New  Mexico.  The  NRL 
ground-based  Multi-sensor  Towed  Array  Detection  System  (gMTADS)  provided  baseline 
data  for  comparison.  All  three  systems  employed  cesium-vapor,  total-field 
magnetometers  (Geometries  Model  822  for  the  two  MTADS  and  Scintrex  CS-2  for 
ORAGS)  but  differed  in  platform  setup  and  data  processing. 

■  gMTADS  employed  a  low-magnetic-signature  vehicle  to  tow  a  linear  array  of 
magnetometers.  It  had  the  smallest  sensor-to-sensor  (25  cm)  and  target-to- 
sensor  separation  (nominal  sensor  height  above  ground  was  also  25  cm). 

■  Both  aMTADS  and  ORAGS  employed  similar  Bell  helicopter  models,  but 
aMTADS  mounted  seven  sensors  on  a  forward  boom  with  1.5  m  spacing, 
whereas  ORAGS  deployed  four  sensors  on  a  forward  boom  and  four  on  a 
mid-ship  boom  with  1.75  m  cross-track  spacing.  Both  helicopters  attempted  to 
fly  at  a  nominal  altitude  of  about  2  m. 

■  Oak  Ridge  used  automatic  target  recognition  algorithms  to  process  the 
magnetometer  data  and  classify  targets.  NRL  relied  on  human  judgment  aided 
by  computer-implemented,  dipole-fit  routines; 

PROCEDURE 

The  Institute  for  Defense  Analyses  calculated  probability  of  detection, 
background  alarm  rates,  and  other  appropriate  statistics  for  the  NRL  gMTADS,  NRL 
aMTADS,  and  ORNL  ORAGS  systems  through  three  major  methodologies. 

■  112  ordnance  items  comprising  60  mm  and  81  mm  mortars,  105  mm  shells, 
and  2.75  in.  rockets  were  buried  among  existing  clutter  and  UXO  in  the 
former  training  range. 
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■  A  dig  list  of  unknown  items  was  generated  using  gMTADS  data,  and  the 
unknown  items  were  subsequently  dug  and  identified. 

■  A  dig  list  of  unknown  items  was  generated  using  data  from  the  airborne 
systems,  and  the  unknown  items  were  subsequently  dug  and  identified. 

Each  system  operator  provided  an  ordered  list  of  detections  and  assigned  each 
detection  a  “confidence  of  ordnance”  ranging  from  1  being  “high  confidence  UXO”  to  6 
being  “high  confidence  clutter.” 

The  criteria  for  the  1-6  classification  were  determined  by  each  operator  utilizing 
metrics  considered  appropriate  for  his  system.  The  digging  of  detected  unknown  items 
provided  additional  statistics,  which  were  analyzed  separately  from  the  seeded  ordnance. 
The  ability  of  the  three  systems  to  successfully  detect  and  identify  the  buried  and  dug 
ordnance  was  then  measured  and  appropriate  receiver  operating  characteristics  (ROC) 
curves  calculated. 

FINDINGS  AND  CONCLUSIONS 

A  priori  consideration  of  the  hardware  similarity  between  aMTADS  and  ORAGS 
would  suggest  similar  overall  performance.  However,  in  spite  of  that  similarity,  their 
differing  platform  setup  and  data-processing  methods  produced  different  results,  with 
aMTADS  performing  better  than  ORAGS  in  testing  against  emplaced  ordnance.  As 
expected,  the  vehicle-towed  system  outperformed  both  airborne  systems  in  the  areas  that 
it  surveyed.  For  both  airborne  systems,  the  detection  performance  demonstrates  that  the 
~1 /(distance)  dependence  of  the  signal  strength  converts  the  increased  distance 
(compared  with  a  ground-based  system)  between  the  target  and  the  sensor  into  a 
challenging  problem.  Based  on  the  results,  we  make  the  following  observations: 

■  Both  systems,  with  their  current,  associated,  data-processing  techniques,  are 
capable  of  reliably  detecting  areas  of  high  clutter  and  potential  UXO  density. 

■  If  maximum  sensitivity  is  desired,  it  is  important  that  an  airborne  system 
design  allow  very  low-level  flight  in  areas  where  that  is  possible. 

■  While  the  ground-based  system  detected  every  emplaced  ordnance  item 
within  its  actual  survey  area,  neither  helicopter-based  system  demonstrated 
reliable  detection  of  small  ordnance  (25%  probability  of  detection  or  less 
against  60  mm  mortar  rounds).  In  addition,  aMTADS  detected  only  three- 
fourths  of  the  105  mm  rounds  and  ORAGS  detected  less  than  one-third  of 
those  rounds.  Those  detection  statistics  would  argue  against  the  use  of  the 
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helicopter  systems  for  individual  UXO  detection,  except  for  large  ordnance  in 
benign  topography,  land  cover,  and  magnetic  background  situations. 

■  aMTADS’s  consistent  ability  to  more  accurately  determine  target  position 
(33  cm  average  radial  error  on  the  emplaced  targets  vs.  97  cm  for  ORAGS) 
improves  target  reacquisition  performance  and  reduces  ambiguity 
opportunities. 

■  Current  automated  target  detection  and  recognition  algorithms  do  not  appear 
to  be  robust  or  accurate  enough  to  replace  the  use  of  human  judgment  in 
interpreting  the  magnetometer  data  and  determining  which  targets  are  likely 
UXO. 
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I.  INTRODUCTION 


Under  Contract  DASW01-04-C-0003,  Task  AM-2-1528,  the  Institute  for  Defense 
Analyses  (IDA)  supports  the  Strategic  Environmental  Research  and  Development 
Program  (SERDP)  and  the  Environmental  Security  Technology  Certification  Program 
(ESTCP)  with  analyses  of  emerging  technologies  in  the  unexploded  ordnance  (UXO) 
detection  and  discrimination  areas  and  with  independent  evaluations  of  system 
performance  in  technology  demonstrations.  The  purpose  of  this  document  is  to  provide 
the  results  of  IDA’s  evaluation  of  data  collected  by  the  Naval  Research  Laboratory 
(NRL)  airborne  and  ground-based  magnetometer  systems  and  the  Oak  Ridge  National 
Laboratory  (ORNL)  airborne  magnetometer  system  in  testing  conducted  at  Isleta  Pueblo, 
New  Mexico,  during  February  2003.  We  first  give  a  brief  description  of  the  test  areas  and 
the  data  collected.  We  then  summarize  the  detailed  analysis  and  provide  data  examples  to 
allow  the  reader  to  understand  the  basis  upon  which  findings  and  conclusions  are  drawn. 

A.  ISLETA  TEST  SITE 

The  area  of  Isleta  Pueblo  chosen  for  survey  is  a  former  practice  bombing  range 
for  Kirtland  AFB  that  was  in  active  use  from  1956-1961.  Previous  surveys  and  surface- 
clearance  activities  had  identified  a  “bull’s-eye”  area  with  a  large  concentration  of 
ordnance  debris.  Figures  1  and  2  provide  a  residual  magnetic  field  map  and  an  analytic 
signal  map  of  the  area,  where  the  bull’s-eye  is  clearly  visible.  Figure  3  is  a  topographic 
map  with  the  individual  survey  areas  indicated  by  boxes.  All  three  systems  surveyed 
approximately  20  hectares  in  the  vicinity  of  the  bull’s-eye  area  (indicated  as  the  vehicular 
area  on  Figure  3),  but  the  bull’s-eye  area  itself  was  excluded  from  scoring  because  the 
debris  density  was  considered  too  high  for  unambiguous  association  of  dug  items  with 
specific  detections  or  for  accurate  discrimination.  A  total  of  112  ordnance  items 
comprising  60  mm  and  81  mm  mortars,  105  mm  shells,  and  2.75-inch  rockets  were 
buried  within  the  joint  survey  area.  Both  the  NRL  airborne  Multi-Sensor  Towed  Array 
Detection  System  (aMTADS)  and  the  Oak  Ridge  Airborne  Geophysics  System  (ORAGS) 
surveyed  a  total  area  of  about  600  hectares,  including  all  of  the  area  around  the  bull’s-eye 
that  included  the  seeded  items.  Post-survey  digs  were  completed  at  more  than  400 
declared  detection  locations  to  provide  additional  truth  data  for  scoring.  For  the  purpose 
of  this  analysis,  we  have  divided  the  site  into  two  areas,  one  where  all  three  systems 
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conducted  surveys  and  one  where  only  the  two  airborne  systems  surveyed.  Unknown 
item  dig  lists  were  produced  for  both  areas.  Details  on  how  dig  items  were  selected  are 
provided  in  the  data  description  section. 
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Figure  1.  aMTADS  Residual  Magnetic  Field  Map  of  the  Isleta  Survey  Area 
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Figure  2.  ORAGS  Analytic  Signal  Map  of  the  Isleta  Survey  Area 
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Figure  3.  Topographic  Map  of  the  Isleta  Survey  Area  with  the  Various  Survey  Areas 

Marked  by  Boxes  [1] 
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B.  SENSOR  SYSTEM  DESCRIPTIONS  [1,  2] 


Three  magnetometer  systems  deployed  to  Isleta  to  collect  anomaly  data.  The 
primary  reason  for  the  testing  was  to  evaluate  the  performance  of  two  airborne  systems 
that  are  capable  of  providing  rapid,  wide-area  surveys  in  benign  topography/ground  cover 
regions.  The  NRL  vehicle -towed  MTADS,  here  denoted  gMTADS  and  shown  in  Figure 
4,  was  used  as  the  “reference”  collection  platfonn  for  a  part  of  the  area  because  it  has 
better  sensitivity  than  the  airborne  systems.  The  gMTADS  employs  eight  cesium-vapor, 
total-field  magnetometers  spaced  25  cm  in  the  across-track  direction.  The  sensor  sample 
rate  is  50  Hz,  which,  combined  with  a  typical  vehicle  speed  of  3  m/s,  results  in  a  6  cm 
down-range  sample  spacing.  The  major  advantage  of  the  vehicle-towed  system,  however, 
is  its  nominal  25  cm  sensor-to-ground  standoff.  With  the  static  magnetic  signal  strength 
falling  off  at  a  l/(distance)  rate,  the  much  smaller  sensor-to-target  separation  for  the 
gMTADS  translates  to  much  stronger  signal  strengths.  The  lesser  separation  also  results 
in  smaller  individual  sensor  footprints  and  thus  in  less  area  averaging  of  the  detected 
signal. 


Figure  4.  NRL  Multi-Sensor  Towed  Array  Detection  System  (gMTADS) 

The  original  test  plan  called  for  the  gMTADS  to  survey  40  hectares  in  the  vicinity 
of  the  bull’s-eye.  A  system  mechanical  breakdown  during  the  survey  resulted  in  coverage 
of  just  over  half  the  planned  area.  Because  the  area  not  covered  included  some  of  the 
emplaced  items,  scoring  of  the  gMTADS  takes  into  account  only  the  actual  area  that  it 
covered. 
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Two  helicopter-borne  sensor  arrays  were  demonstrated  in  the  tests  at  Isleta.  Each 
was  carried  on  a  similar  Bell  206L  Long  Ranger  helicopter.  The  arrays  employed  cesium- 
vapor,  total  field  magnetometers  (Geometries  Model  822  for  aMTADS  and  Scintrex  CS- 
2  for  ORAGS)  mounted  on  nonmetallic  booms  and  differential  GPS  systems  to  provide 
accurate  position  data  for  anomaly  location  and  for  discrimination  algorithm  inversion. 
The  arrays,  navigation  instrumentation,  and  data  processing  differed  in  a  number  of 
details,  however. 

ORAGS,  developed  by  ORNL  and  shown  in  Figure  5,  deployed  the 
magnetometer  sensors  on  the  “Arrowhead”  array.  This  array  included  four  outboard 
magnetometers,  two  on  each  side  of  the  aircraft,  mounted  on  an  amidships  horizontal 
boom,  and  four  magnetometers  spaced  horizontally  on  a  boom  forward  of  the  helicopter 
nose.  Magnetometer-to-magnetometer  spacing  was  1.75  m.  While  this  arrangement 
provided  a  very  wide  array  (12.25  m  array  width  and  potentially  a  14  m  survey  swath), 
the  fore-aft  spacing  of  the  sensors  and  the  wide  lateral  spread  complicates  signal  leveling. 
The  ORAGS  data-collection  system  sampled  at  a  1,200  Hz  rate,  but  commonly  the  data 
are  downsampled  to  120  Hz  for  analysis.  In  the  data-analysis  phase,  the  ORNL  team 
created  an  analytic  signal  map  and  then  employed  an  automated  threshold  exceedence 
algorithm  to  pick  peaks  for  analysis.  Two  automatic  algorithms  were  used  for  detection 
and  classification  analysis  of  the  Isleta  data. 


Figure  5.  Oak  Ridge  Airborne  Geophysics  System  (ORAGS) 

The  airborne  version  of  the  MTADS,  which  we  designate  aMTADS  and  show  in 
Figure  6,  is  an  outgrowth  of  the  vehicle-towed  system.  It  employed  seven  magnetometers 
on  a  single  horizontal  boom  in  front  of  the  helicopter  nose.  Magnetometer  spacing  was 
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1.5  m  (9  m  array  width  and  potentially,  a  10.5  m  swath),  and  sensor  sampling  was 
accomplished  at  100  Hz.  The  NRL  team  handpicked  anomalies  from  the  total  field  maps 
and  then  applied  a  dipole-fit  analysis  in  selecting  and  classifying  targets. 


Figure  6.  NRL  Airborne  MTADS  (aMTADS) 

Both  aircraft  attempt  to  maintain  an  altitude  of  a  few  meters  above  the  ground  and 
a  forward  speed  of  around  20  m/s  (aMTADS  operated  at  about  10  m/s  at  Isleta).  The 
20  m/s  speed  results  in  nominal  down-track  sample  spacings  of  15-20  cm,  which  is  much 
finer  than  the  cross-track  sampling  for  either  magnetometer  array.  Cross-track  sampling 
finer  than  the  1.5  m  or  1.75  m  sensor  spacing  can  be  provided  by  multiple  passes,  or 
overlapping  passes,  at  the  expense  of  area  coverage  rate.  NRL  typically  flies  7  m  lane 
spacings  with  a  nominal  10.5  m  array  footprint,  to  prevent  holes  in  coverage,  while 
ORNL  uses  12  m  lane  spacings,  for  their  14  m  array  footprint. 


C.  DATA  DESCRIPTION 

IDA  received  several  basic  types  of  data  for  this  system  performance  evaluation 
effort.  The  first  was  a  list  of  ordnance  buried  within  the  three-system  section  by 
personnel  from  the  Anny  Engineering  R&D  Center  (ERDC),  Vicksburg,  Miss.  There  was 
a  coordinate  conversion  problem  in  the  initial  list  of  calibration  objects  provided  by 
ERDC  to  the  demonstrators,  and  both  teams  flew  the  calibration  lanes  based  on  the 
incorrect  coordinates.  Errors  in  calibration  target  position  varied  from  about  5  m  to  25  m. 
ERDC  subsequently  corrected  the  list,  and  the  corrected  list  was  provided  to  the 
demonstrators  before  dig  lists  were  submitted.  ERDC  rechecked  the  coordinates  of  all  the 
buried  seed  items  when  they  were  recovered  (calibration  and  blind  area),  and  maximum 
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differences  between  pre-burial  and  post-dig  position  data  were  typically  no  greater  than  a 
few  centimeters.  The  second  set  of  data  was  detection  lists  from  the  three  participating 
systems.  Although  each  list  provided  ancillary  information,  the  important  data  items  for 
the  analysis  were  the  estimated  position  of  each  anomaly  and  the  likelihood  that  the 
anomaly  represented  UXO.  The  standard  six-category  division  of  UXO  likelihood  was 
used,  where  category  1  represents  most  likely  UXO  and  category  6  represents  most  likely 
clutter. 

An  ESTOP,  HydroGeoLogic  Inc.  (ESTOP  support  contractor),  and  IDA  team 
used  the  system  detection  lists  to  generate  separate  dig  lists  for  the  area  covered  by  all 
three  systems  and  for  the  airborne-only  area.  The  three-system  list  contained  all 
gMTADS  category  1  and  2  detections  plus  some  large  items  from  the  aMTADS  survey 
assessed  to  be  of  interest.  From  that  dig  list,  a  total  of  272  items  were  dug  and 
categorized  as  intact  ordnance,  ordnance-related  scrap,  non-ordnance-related  clutter, 
geology,  or  as  an  empty  hole.  In  the  area  covered  by  only  the  airborne  systems,  all 
category  1  and  2  items  for  both  systems  were  on  the  dig  list.  The  dig  resulted  in  161 
items  for  scoring. 

For  the  three-systems  area,  gMTADS  coordinates  were  provided  to  the  dig  team 
because  of  their  tighter  error  statistics.  For  the  airborne-only  area,  there  were  two  generic 
cases.  Anomalies  on  one  list  either  had  a  match  on  the  other  list  or  did  not  (i.e.,  the 
position  for  an  anomaly  on  the  other  list  either  fell  within  a  1.5  m  halo  or  did  not).  For 
the  no-match  case,  the  dig  team  received  the  coordinates  from  the  anomaly  list  on  which 
the  target  appeared.  Where  items  had  a  match,  the  aMTADS  (rather  than  ORAGS) 
coordinates  were  used,  again  because  of  that  system’s  better  geolocation  statistics  against 
emplaced  ordnance.  While  there  was  a  possibility  that  choice  could  bias  results,  use  of 
the  most  accurate  available  location  data  was  felt  essential  in  the  heavily  cluttered 
environment  at  Isleta.  Since  in  both  cases  demonstrators  were  scored  against  the  actual 
dug  locations  of  the  recovered  items,  and  not  against  the  coordinates  provided  in  the  dig 
lists,  this  procedure  fairly  measures  the  probability  that  either  system  detected  a 
particular  item  that  was  dug. 

At  ESTCP  direction,  NRL  was  required  to  submit  an  aMTADS  dig  list  for  the 
three-system  area  before  data  collection  by  gMTADS.  Weather  delays  in  collecting  the 
aMTADS  data  compressed  the  analysis  schedule  to  such  an  extent  that  the  NRL  team 
delivered  an  incomplete  analysis,  principally  for  areas  on  the  western  end  of  the  survey. 
After  noting  that  analysis  had  not  been  completed  for  important  areas  of  the  site,  ESTCP 
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directed  that  NRL  submit  a  second,  more  complete,  detection  list.  IDA  denoted  that  list 
“aMTADSE”  for  aMTADS-extended  and  used  that  list  for  the  scoring. 

Finally,  after  IDA’s  production  of  a  draft  memorandum  on  the  demonstration 
results,  both  teams  provided  position  and  altitude  data  for  portions  of  the  testing  to  allow 
an  analysis  of  the  speeds  and  altitudes  flown  by  the  two  systems.  Those  data  were  used  to 
assess  how  the  way  in  which  the  systems  were  operated  might  have  affected 
performance. 
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II.  ANALYSIS 


A.  OVERVIEW 

IDA  has  assessed  detection  perfonnance  and  location-error  statistics  for  the 
emplaced  ordnance,  for  the  dug  items  in  the  area  covered  by  all  three  systems,  and  for  the 
dug  items  in  an  area  covered  only  by  the  airborne  systems  (denoted  the  Primary  Area  in 
Figure  3).  We  have  also  assessed  discrimination  performance,  based  on  the  three-system 
dig  results.  Discrimination  performance  is  discussed  in  section  II. C  and  in  Appendix  C. 

One  immediate  question  in  scoring  performance  concerns  the  size  and  shape  of 
the  area  around  an  item  within  which  a  declaration  is  counted  as  a  detection.  For  this 
analysis,  we  use  a  circular  area,  denote  its  perimeter  as  the  detection  “halo,”  and  define  it 
in  terms  of  the  radial  distance  from  the  center  of  the  target.  From  a  clearance  point  of 
view,  the  more  accurately  a  system  locates  an  item  to  be  dug,  the  more  efficiently  it  can 
be  reacquired  for  digging,  and  the  less  likely  it  is  to  be  confused  with  a  nearby  item.  If 
discrimination  improves  to  the  point  that  only  items  judged  likely  to  be  ordnance  are 
excavated,  accurate  location  capability  will  become  even  more  important.  On  the  other 
hand,  to  fully  assess  system  performance,  the  radius  should  be  large  enough  to  adequately 
represent  system-detection  capabilities. 

To  assess  halo  radii  for  analysis,  we  looked  at  detection  results  for  the  two 
airborne  systems  as  a  function  of  halo  area.  Because  the  percentage  of  total  detections 
made  by  chance  begins  to  rise  rapidly  as  the  total  area  of  detection  calls  becomes  a 
significant  percentage  of  the  survey  area,  a  pure  detection  analysis  is  necessarily 
incomplete.  Reference  3  contains  a  development  that  provides  an  estimate  of  “true”  and 
“chance”  detections  based  on  the  survey  area,  halo,  total  number  of  declarations,  and 
total  number  of  detections.  Appendix  A  provides  a  brief  explanation  of  the  analysis  and 
contains  the  resulting  equations  for  true  and  chance  detections.  If  we  assume  a  detection 
system  exhibits  random,  zero-mean,  location  errors,  the  curve  for  true  detections  would 
be  expected  to  flatten  out  as  system  location  error  limits  are  reached  and  additional 
detections  became  all  chance  detections.  Figure  7  provides  such  a  set  of  curves  for  the 
Isleta  emplaced  ordnance.  We  have  plotted  the  probability  of  detection  (Pd)  for  the  total 
number  of  detections  for  each  airborne  system;  the  “true”  Pd  based  on  the  chance 


detection  analysis  of  Appendix  A;  and  the  percentage  of  the  total  survey  area  each 
system’s  declarations  would  have  covered  for  that  halo,  ignoring  overlap  and  the  area 
outside  the  boundaries  of  the  survey  area. 


Figure  7.  Pd  vs.  Halo  Radius  for  the  Isleta  Emplaced  Ordnance 

Note  that  the  aMTADSE  “true”  Pd  curve  behaves  as  expected,  flattening  out 
between  a  1.0  and  1.5  m  halo.  The  ORAGS  “true”  Pd  curve,  in  contrast,  increases  until  it 
reaches  a  2.25  m  halo,  after  which  Pd  begins  to  decrease.  Clearly,  true  Pd  cannot 
decrease,  and  the  fact  that  the  curve  turns  over  is  an  indication  that  the  assumptions  of  the 
theory  are  being  violated.  At  the  point  of  inflection,  over  one -third  of  the  survey  area 
would  be  included,  assuming  no  overlap.  For  such  large  percentages  of  area  coverage, 
there  will  likely  be  significant  overlap  of  halos.  The  result  of  overlap  is  that  actual  chance 
detections  do  not  go  up  as  rapidly  as  the  theory  predicts,  providing  an  apparent  decrease 
in  true  detections. 

Obviously,  to  show  aMTADS  in  its  relatively  best  light  we  should  pick  no  larger 
than  a  1.0  m  halo,  and  to  show  ORAGS  in  its  relatively  best  light  we  should  pick  a  2.0  or 
2.25  m  halo.  For  most  of  the  results  here,  we  have  used  the  1.5  m  halo  as  a  reasonable 
compromise,  one  that  does  not  provide  a  significant  relative  advantage  to  either  system 
and  for  which  the  spread  between  true  and  total  detections  is  not  too  great.  Where 
appropriate,  however,  we  present  1.0  m  and  2.0  m  halo  results  to  emphasize  system 
performance  differences  as  halos  change. 
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B.  EMPLACED  ITEMS 


The  ERDC-emplaced  items  were  blind  for  the  gMTADS  demonstrators,  so 
gMTADS,  as  well  as  ORAGS  and  aMTADS,  was  scored  for  those  items.  Table  1 
provides  the  number  of  each  emplaced  ordnance  type  detected  for  a  1.5  m  halo.  ORAGS 
and  aMTADSE  surveyed  the  entire  seeded  area  and  are  scored  on  the  total  number  of 
items  and  total  area.  Because  of  the  mechanical  breakdown,  gMTADS  did  not  cover  the 
entire  seeded  area,  so  its  scoring  is  limited  to  the  items  within  the  boundaries  that  it 
surveyed.  The  totals  on  which  it  is  scored  are  reflected  in  its  column  in  the  table. 


Table  1.  Emplaced  Ordnance  Detection  by  Type  for  a  1.5  m  Halo 


Ordnance 

Total 

ORAGS 

aMTADSE 

gMTADS 

2.75  inch 

12 

7 

11 

2  of  2 

60  mm 

20 

5 

4 

6  of  6 

81  mm 

40 

15 

19 

20  of  21 

105  mm 

40 

15 

29 

17  of  18 

Total 

112 

42 

63 

45  of  47 

To  achieve  the  detections  shown  in  the  table,  ORAGS  declared  1,937  targets, 
aMTADSE  declared  165  targets,  and  gMTADS  declared  104  targets.  Receiver  operating 
characteristic  (ROC)  curves  were  produced  for  each  system  using  the  six  ordnance 
likelihood  categories.  Pd  is  plotted  vs.  background  alarm  rate  (in  number  of  alanns  per 
hectare),  where  the  survey  area  for  the  airborne  systems  was  9.63  hectares  and  for 
gMTADS  was  3.71  hectares.  Note  that  a  substantial  number  of  items  counted  as 
background  alarms  are  likely  ordnance.  From  the  dug  items  in  the  three-system  area, 
which  were  based  on  the  gMTADS  high-probability  UXO  calls  (categories  1  and  2), 
about  24%  (65  of  272)  of  the  items  recovered  were  intact  ordnance.  Thus,  we  have  not 
labeled  the  abscissa  as  false-alarm  rate  because  a  substantial  number  of  the  background 
alarms  are  likely  ordnance  and  not  false-alarms  at  all.  Figures  8  and  9  provide  the  results 
for  the  1.5  m  halo  and  the  1.0  m  halo.  In  going  from  a  1.5  to  1.0  m  halo,  gMTADS  lost 
no  targets,  aMTADSE  lost  1  target,  and  ORAGS  lost  24  of  42  targets. 
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Figure  8.  ROC  Curve  for  Isleta  Emplaced  Items  and  a  1.5  m  Halo  Including  60  mm 


1.0  m  halo 


— aMTADSE 
— gMTADS 
-A-ORAGS 


Figure  9.  ROC  Curve  for  Isleta  Emplaced  Items  and  a  1.0  m  Halo  Including  60  mm 
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The  airborne  systems  were  not  designed  to  routinely  detect  ordnance  smaller  than 
8 1  mm,  so  similar  ROC  curves  were  produced  with  the  60  mm  targets  removed  from  the 
scoring.  Because  gMTADS  detected  all  the  60  mm  rounds,  removing  them  actually  hurt 
its  ROC  curve.  ORAGS  detected  5  of  20  and  aMTADSE  4  of  20  emplaced  60  mm,  so 
removing  the  60  mm  improved  their  overall  Pds  almost  equally.  Figure  10  shows  the 
detection  results  for  the  three  systems  for  a  1.5  m  halo  and  the  case  where  60  mm  are  not 
considered.  Figure  1 1  shows  the  same  results  for  the  three  systems  for  a  2.0  m  halo. 


Figure  10.  ROC  Curves  for  Isleta  Emplaced  Ordnance  Excluding  60  mm  Targets  for  a  1.5 

m  Halo 
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Figure  11.  ROC  Curves  for  Isleta  Emplaced  Ordnance  Excluding  60  mm  Targets  for  a  2.0 

m  Halo 

The  difference  between  automatic  and  manual  anomaly  selection  created  a  large 
difference  in  the  number  of  detection  calls  between  the  systems.  It  is  therefore  useful  to 
apply  the  chance  detection  analysis  of  Appendix  A  to  detennine  what  part  that  might 
have  played  in  the  detections.  Table  2  shows  cumulative  true  and  chance  detections  based 
on  the  1.5  m  halo  matches.  The  formula  in  Appendix  A  does  not  provide  integer  results, 
so  values  are  rounded  to  the  nearest  whole  number.  Because  of  the  relatively  few  total 
declarations  by  the  two  MTADS  systems,  the  analysis  indicates  that  none  of  the 
detections  provided  by  those  systems  were  likely  chance  detections.  For  ORAGS,  a  total 
of  only  223  detection  calls  were  made  in  the  first  3  ordnance  likelihood  categories,  and 
the  analysis  indicates  that  17  of  18  of  the  matches  to  emplaced  ordnance  were  likely  true 
detections.  Categories  4-6  contain  a  great  majority  of  the  total  number  of  ORAGS 
detection  calls  (1,714  of  the  1,937),  and  a  substantial  number  of  the  ordnance  found  in 
those  categories  were  likely  chance  detections  (3  of  7  in  category  4,  3  of  1 1  in  category  5, 
and  4  of  6  in  category  6). 
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Table  2.  Cumulative  Total  and  True  Detections  for  the  Emplaced  Ordnance  and  1.5  m  Halo 


UXO  Likelihood 

ORAGS  detections 

Total/True 

aMTADS  detections 

Total/True 

gMTADS  detections 

Total/True 

1 

2/2 

24/24 

20/20 

2 

6/6 

44/44 

32/32 

3 

18/17 

60/60 

42/42 

4 

25/21 

61/61 

44/44 

5 

36/29 

62/62 

45/45 

6 

42/31 

63/63 

45/45 

To  further  explore  the  results  of  Table  2,  IDA  analyzed  ORAGS  emplaced 
ordnance  detections  by  ordnance  type  and  UXO  likelihood  category.  Of  the  five  60  mm 
mortars  detected  by  ORAGS,  one  was  a  category  2,  three  were  category  5,  and  one  was 
category  6.  For  the  fifteen  detected  81  mm  mortars,  one  was  a  category  3,  five  were 
category  4,  five  were  category  5,  and  four  were  category  6.  Thus,  18  of  the  20  detected 
60  and  81  mm  mortars  were  in  categories  4-6.  Note  from  Table  2  that  chance  detection 
theory  indicates  that  10  of  the  24  ORAGS  detections  of  emplaced  UXO  in  categories  4-6 
were  likely  chance  detections.  Based  on  those  numbers,  about  40%  of  the  60  and  81  mm 
detections  were  likely  by  chance.  In  contrast,  only  six  of  the  22  ORAGS  detections  of 
105  mm  mortars  and  2.75  inch  rockets  fell  in  categories  4-6. 

Figures  12-14  provide  location-error  scatter  plots  for  the  three  systems  where  the 
1.5  m  halo  data  have  been  used.  The  ORAGS  locations  are  biased  23  cm  north  and  22  cm 
west,  with  error  standard  deviations  of  81  cm  and  56  cm  in  each  direction.  The  aMTADS 
locations  are  biased  4  cm  north  and  4  cm  west,  with  standard  deviations  of  3 1  cm  and 
32  cm.  The  gMTADS  locations  are  biased  4  cm  south  and  1  cm  east,  with  standard 
deviations  of  13  cm  and  12  cm. 

Figures  15-17  provide  similar  data  for  radial  location  error,  which  is  defined  as 
the  square  root  of  the  sum  of  the  squares  of  the  east  and  north  errors.  For  the  emplaced 
items,  ORAGS  demonstrated  an  average  radial  location  error  of  97  cm,  and  aMTADSE 
showed  a  33  cm  average  radial  error  for  those  targets.  In  contrast,  gMTADS  average 
radial  error  was  only  13  cm. 
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Figure  12.  ORAGS  Location  Error  Scatter  Plot  for  Isleta  Emplaced  Items  and  a  1.5  m  Halo 
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Figure  13.  aMTADSE  Location  Error  Scatter  Plot  for  Isleta  Emplaced  Items  and  a 

1.5  m  Halo 
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Figure  14.  gMTADS  Location  Error  Scatter  Plot  for  Isleta  Emplaced  Items  and  a  1.5  m  Halo 
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Figure  15.  ORAGS  Radial  Location  Error  Histogram  for  Isleta  Emplaced  Items  and  a 

1.5  m  Halo 
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Figure  16.  aMTADSE  Radial  Location  Error  Histogram  for  Isleta  Emplaced  Items  and  a 

1.5  m  Halo 
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Figure  17.  gMTADS  Radial  Location  Error  Histogram  for  Isleta  Emplaced  Items  and  a 

1.5  m  Halo 
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C.  THREE-SYSTEM  AREA  PERFORMANCE 


The  second  set  of  performance  analyses  was  undertaken  on  the  area  that  was 
jointly  covered  by  all  three  systems.  Based  on  the  detection  lists  produced  by  each 
system,  a  joint  dig  list  was  fonned.  The  gMTADS  was  effectively  used  as  the  reference 
to  establish  the  dig  list,  which  included  all  of  the  category  1  and  2  picks  for  gMTADS, 
plus  some  detections  highlighted  as  very  large  items.  Thus,  Pd  for  the  three-system  area 
is  the  probability  that  a  system  detected  dug  items  initially  classified  by  the  gMTADS  as 
likely  ordnance.  Even  for  targets  providing  matches  from  all  three  systems,  the  dig  list 
used  gMTADS  target  locations  because  of  the  demonstrated  superior  geolocation 
performance  of  that  sensor.  In  the  test,  272  items  were  dug  and  sorted  into  5  classes: 
intact  ordnance,  ordnance-related  scrap,  non-ordnance-related  clutter,  geology,  and 
empty  hole.  A  match  was  then  perfonned  against  the  three  detection  lists,  using  actual 
dug  coordinates  of  the  recovered  items,  not  the  coordinates  provided  to  the  dig  team.  The 
results:  260  of  those  items  matched  gMTADS  category  1  or  2  items,  1  matched  a 
gMTADS  category  4  item,  and  1 1  matched  no  gMTADS  item.  Five  digs  had  no  match  to 
any  of  the  three  systems.  In  every  case  for  the  five  unmatched  items,  there  was  a 
declaration  within  the  halo  of  the  dug  item,  but  the  declaration  was  closer  to  another  dug 
item  and  was  associated  with  it  instead.  These  misses  apparently  resulted  from  cases 
where  more  than  one  object  was  detected  by  the  dig  team  in  the  vicinity  of  a  dig  list 
coordinate,  and  more  than  one  item  was  recovered  and  listed  in  the  database.  Because  of 
the  unmatched  items,  even  gMTADS,  the  reference  system  for  the  dig,  did  not  achieve  a 
Pd  of  unity.  Table  3  provides  the  number  of  items  in  each  class,  sorted  by  the  gMTADS 
discrimination  category  into  which  it  fell.  Note  that  there  were  no  empty  holes  in  this 
case. 


Table  3.  Three-System  Area  Dug  Items  by  Category  and  gMTADS  UXO 

Likelihood  Category 


UXO 

Likelihood 

Intact 

Ordnance 

Ordnance- 
related  scrap 

Non- 

Ordnance- 
related  clutter 

Geology 

Empty 

hole 

1 

54 

159 

7 

1 

0 

2 

6 

27 

4 

2 

0 

4 

0 

1 

0 

0 

0 

No  Match 

5 

5 

0 

1 

0 

Total 

65 

192 

11 

4 

0 
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Two  classes  in  Table  3  relate  to  ordnance.  In  scoring  system  perfonnance,  we 
have  chosen  to  calculate  Pd  and  background  alarm  rates  two  ways.  First,  we  count  only 
intact  ordnance  in  Pd;  ordnance-related  scrap  detections  count  in  the  false-alarms  (case 
1).  Second,  we  group  intact  ordnance  and  ordnance-related  scrap  into  Pd;  non-ordnance- 
related  clutter  and  geology  detections  contribute  to  the  false-alarm  rate  (case  2).  In  both 
cases,  the  dug  items  are  not  included  in  the  background  alann  rate.  Figure  18  provides 
both  sets  of  calculations  for  the  1.5  m  halo;  Figure  19  provides  data  for  a  2.0  m  halo.  The 
detections  and  background  alarm  rates  are  based  on  6,676  declarations  by  ORAGS,  1,136 
by  aMTADSE,  and  1,237  by  gMTADS. 
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Figure  18.  Pd  vs.  Background  Alarm  Rate  for  the  Three-System  Area  and  a  1.5  m  Halo 
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3-system  area,  2.0  m  halo 
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Figure  19.  Pd  vs.  Background  Alarm  Rate  for  the  Three-System  Area  and  a  2.0  m  Halo 

We  also  completed  an  analysis  of  location-error  statistics.  Table  4  provides  the 
mean  location  errors  and  standard  deviations  for  each  system.  Figure  20  provides  scatter 
plots  of  those  data  for  the  1.5  m  halo  detections.  We  also  found  it  of  interest  to 
investigate  how  the  radial  errors  were  distributed,  and  Figure  21  provides  histograms  of 
those  data  for  the  three  systems,  again  for  the  1.5  m  halo. 


Table  4.  Location  Error  Statistics  for  the  Three-System  Area  and  1 .5  m  Halo 


Mean  Error  (cm) 

Error  Std.  Dev  (cm) 

System 

North 

East 

North 

East 

ORAGS 

45 

-1 

60 

61 

aMTADSE 

2 

6 

39 

35 

gMTADS 

1 

6 

26 

25 
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Figure  20.  Scatter  Plot  of  Location  for  the  Three-System  Area  and  1.5  m  Halo  for  ORAGS 
(top),  aMTADSE  (middle),  and  gMTADS  (bottom) 
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Figure  21 .  Radial  Location  Error  Histograms  for  the  Three-System  Area  and  1 .5  m  Halo  for 
ORAGS  (top),  aMTADSE  (middle),  and  gMTADS  (bottom) 


11-15 


Finally,  we  used  the  three-system  data  to  evaluate  discrimination  perfonnance  for 
all  systems.  Table  5  gives  the  airborne  system  results,  cross-referenced  for  the  gMTADS 
category  1  matches  in  the  tables  in  Appendix  C.  Because  the  dig  list  concentrated  on 
gMTADS  category  1  and  2  items,  we  would  have  hoped  that  the  dig  results  would  be 
heavily  biased  toward  intact  UXO.  Note  from  Table  3,  however,  that  only  65  of  the  272 
recovered  items  were  intact  ordnance.  On  the  other  hand,  for  the  items  matched  to 
gMTADS  category  1  or  2  calls,  only  14  of  the  260  items  were  non-ordnance-related 
clutter  (1 1)  or  geological  (3)  in  nature.  Thus,  the  gMTADS  appeared  to  show  an  excellent 
capability  for  discriminating  ordnance-related  detections  from  non-ordnance-related 
ones,  albeit  with  much  less  capacity  to  discriminate  intact  ordnance  from  ordnance- 
related  scrap. 


Table  5.  Airborne  System  Discrimination  Results  from  the  Three-System  Area  Matches 


System 

Category 

Intact  Ord. 

Ord.  Frag. 

Clutter 

Geology 

ORAGS 

1 

9 

21 

0 

0 

2 

12 

30 

0 

0 

3 

11 

39 

2 

1 

4 

7 

31 

1 

0 

5 

3 

9 

2 

2 

6 

4 

3 

0 

0 

No  Match 

19 

59 

6 

1 

aMTADSE 

1 

44 

85 

4 

1 

2 

12 

48 

1 

1 

3 

5 

35 

1 

0 

4 

2 

1 

0 

0 

5 

0 

2 

0 

0 

6 

0 

0 

0 

0 

No  Match 

2 

21 

5 

2 
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The  aMTADS  uses  the  same  discrimination  algorithms  as  gMTADS,  but  they  are 
applied  against  a  much  less  dense  set  of  magnetometer  sample  points  with  poorer  signal- 
to-noise  ratio  because  of  increased  target-to-sensor  separation.  aMTADS  placed  56  of  the 
65  intact  ordnance  items  in  categories  1  or  2,  with  five  items  in  category  3,  two  in 
category  4,  and  two  not  detected.  Similarly,  133  of  the  192  ordnance  fragment  items  also 
appear  in  categories  1  and  2.  Seven  of  the  fifteen  non-ordnance-related  items  were  not 
detected  by  aMTADS,  but  seven  of  the  eight  detected  items  were  ranked  as  category  1  or 
2.  In  fact,  196  of  the  242  items  matched  by  aMTADS  were  in  categories  1  and  2,  and  41 
more  were  in  category  3,  so  the  classification  algorithm  leaned  heavily  toward  the  more 
likely  ordnance  categories,  as  categories  4  and  5  had  only  five  matches  among  them 
(three  in  category  4  and  two  in  category  5),  and  category  6  had  no  matches. 

ORAGS  detection  reports  were  more  evenly  weighted  over  the  six  categories, 
with  125  detections  in  categories  1-3  and  62  in  categories  4-6.  However,  only  21  of  the 
46  detected  intact  ordnance  items  appeared  in  categories  1  or  2,  and  seven  intact 
ordnance  items  fell  in  categories  5  and  6.  Ordnance  fragment  detections  were 
concentrated  in  categories  1-4,  but  with  relatively  somewhat  fewer  items  in  category  1, 
in  contrast  to  aMTADS.  The  detected  non-ordnance-related  items  were  in  categories  3,  4, 
and  5,  so  there  was  less  tendency  than  in  the  aMTADS  case  to  call  non-ordnance  items 
likely  ordnance  related. 

D.  PRIMARY  (AIRBORNE-ONLY)  AREA 

The  third  set  of  data  analyzed  was  from  the  area  covered  only  by  the  two  airborne 
systems.  A  dig  list  was  generated  from  the  high-priority  (category  1  and  2)  items  for  each 
system.  As  noted  before,  for  items  that  appeared  within  a  halo  of  each  other  for  the  two 
lists,  aMTADS  coordinates  were  used  in  the  dig  list  because  of  the  smaller  location  errors 
that  system  demonstrated  against  the  emplaced  targets.  Obviously,  for  items  without  an 
apparent  match  on  the  other  list,  the  dig  list  used  the  coordinates  provided  by  the 
detecting  system.  As  in  the  three-system  area,  post-dig  matches  were  done  on  the  dug 
coordinates,  not  the  dig  list  coordinates.  Nevertheless,  it  is  possible  that  preferential  use 
of  aMTADS  coordinates  resulted  in  some  residual  bias  in  the  results,  although  the 
resulting  ROC  curves  and  location-error  plots  are  consistent  with  those  based  on  the 
emplaced  items  and  three-system  area. 

A  total  of  161  items  were  dug.  ORAGS  made  10,876  declarations  in  the  area 
scored,  and  aMTADS  made  366  declarations.  Of  those,  215  of  the  ORAGS  declarations 
were  in  categories  1  and  2,  and  164  of  the  aMTADS  calls  fell  in  the  first  two  categories. 
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The  ORAGS  list  contained  matches  for  131  of  the  dug  items,  of  which  94  appeared  in 
categories  1  and  2.  For  aMTADS,  ill  of  the  matching  126  items  were  in  categories  1 
and  2.  Because  there  was  no  real  reference  system,  unlike  in  the  three-system  area,  we 
have  chosen  to  label  ROC  curves  ordinates  “fraction  of  dug  items  matched,”  vice  Pd. 
Figure  22  provides  a  map  of  the  airbome-only  area  with  the  dug  items  and  UXO 
likelihood  category  1  and  2  declarations  from  each  system  indicated. 
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Figure  22.  Airborne-Only  Area  with  Dug  Items  and  Category  1  and  2  Declarations  from  Each  System 


Figures  23  and  24  provide  ROC  curves  for  1.5  m  and  2.0  m  radius  halos.  Scoring, 
as  for  the  three-system  area,  was  done  twice,  once  with  only  intact  ordnance  contributing 
to  detection  (case  1)  and  once  with  ordnance  scrap  also  contributing  (case  2).  Note  that 
we  have  shifted  to  a  logarithmic  scale  for  background  alarm  rate  to  better  display  the 
data. 


Airborne-only,  1.5  m  halo 
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Figure  23.  Airborne-Only  Area  ROC  Curves  for  a  1 .5  m  Halo 


Airborne-only,  2.0  m  halo 
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Figure  24.  Airborne-Only  Area  ROC  Curves  for  a  2.0  m  Halo 
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Figure  25  provides  location  error  scatter  plots  based  on  the  1.5  m  halo  data. 
Figure  26  provides  the  associated  radial  error  histograms,  and  Table  6  provides  the  error 
statistics. 
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Figure  25.  Location-Error  Scatter  Plots  for  the  Airborne-Only  Area,  Based  on  a  1 .5  m  Halo 
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Figure  26.  Radial  Location  Error  Histogram  for  the  Airborne-Only  Area,  Based  on  a  1 .5  m 

Halo  for  ORAGS  (top)  and  MTADS  (bottom) 
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Table  6.  Location  Error  Statistics  for  the  Airborne-Only  Area  and  1.5  m  Halo 


System 

Mean  Error  (cm) 

Error  Std.  Dev  (cm) 

North 

East 

North 

East 

ORAGS 

1 

-22 

58 

62 

aMTADSE 

5 

9 

31 

30 
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III.  FINDINGS  AND  CONCLUSIONS 


A.  FINDINGS 

This  was  the  second  in  a  pair  of  demonstrations  designed  to  test  the  detection  and 
classification  capabilities  of  airborne  magnetometer  systems  over  a  wide  range  of  terrain 
types  and  levels  of  background  clutter.  At  Isleta,  112  UXO  were  emplaced  in  an  area 
containing  substantial  ferrous  clutter  to  provide  known  items  against  which  system 
performance  could  be  judged.  This  report  focuses  on  the  detection  and  discrimination 
performance  of  the  two  helicopter  systems,  although  it  is  also  enlightening  to  compare 
that  performance  to  the  ground-based  system. 

These  two  systems  employ  similar  cesium  vapor  magnetometers  carried  on 
identical  helicopters,  so  we  would  expect  their  inherent  signal  sensitivities  to  be  similar, 
and  that  was  the  case  for  the  calibration  targets  and  airfield  targets  analyzed  in  the  earlier 
Aberdeen  Proving  Ground  report  [5],  However,  as  in  the  earlier  case,  the  aMTADS  flew 
at  lower  altitudes  than  did  the  ORAGS.  Based  on  an  IDA  analysis  of  altitude  data 
provided  by  NRL  and  ORNL,  aMTADS  flew  Isleta  at  an  average  altitude  of  1 .4  m  with  a 
0.5  m  standard  deviation,  and  ORAGS  flew  at  an  average  altitude  of  3.0  m  with  a  1.5  m 
standard  deviation.  Jeff  Gamey  of  ORNL,  in  a  personal  communication  [6],  stated  that 
ORAGS  typically  flew  at  altitudes  between  1.5  and  2.5  m  over  the  areas  where  most  of 
the  scoring  was  done.  In  a  situation  where  signal  strength  is  expected  to  fall  off  as 
l/(distance)  ,  a  26%  difference  in  altitude  results  in  a  factor  of  2  difference  in  signal 
strength,  and  going  from  1.4  m  to  2.5  m  reduces  signal  strength  by  more  than  a  factor  of 
5.  Thus,  as  at  APG,  aMTADS’s  lower  altitude  gave  it  a  signal  strength  advantage  that 
certainly  contributed  to  its  better  detection  perfonnance. 

While  the  difference  between  ORAGS  and  aMTADS  performance  was 
significant,  both  suffered  in  comparison  to  the  vehicle-towed  system.  The  gMTADS 
25  cm  sensor  height  should  give  it  much  better  detection  performance  than  the  helicopter 
sensors,  and  that  proved  to  be  the  case  at  Isleta.  gMTADS  detected  every  one  of  the 
emplaced  ordnance  items  over  which  its  sensor  array  traveled.  In  evaluating  the  two 
missed  items  within  the  gMTADS  survey  area,  it  was  apparent  from  recorded  vehicle 
track  data  that  the  items  fell  in  coverage  gaps  that  were  recognized  and  would  have  been 
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resurveyed,  had  not  the  system  broken  down.  In  contrast,  the  airborne  systems  did  not 
fare  that  well  in  the  cluttered  Isleta  environment,  even  against  the  larger  emplaced  UXO. 
aMTADS  detected  only  19  of  40  81  mm  mortars  and  29  of  40  105  mm  shells,  while 
ORAGS  did  even  more  poorly,  detecting  15  each  of  the  81  mm  and  105  nun  emplaced 
targets.  Neither  helicopter-based  sensor  system  was  initially  designed  to  detect  targets  as 
small  as  60  mm  mortar  shells,  and  the  Isleta  testing  confirmed  that  neither  system  fares 
well  against  such  targets  in  a  cluttered  environment. 

B.  CONCLUSIONS 

Several  conclusions  can  be  drawn  from  the  results  of  these  demonstrations: 

■  Both  systems,  with  their  current,  associated,  data-processing  techniques,  are 
capable  of  reliably  detecting  areas  of  high  clutter  and  potential  UXO  density. 

■  If  maximum  sensitivity  is  desired,  it  is  important  that  an  airborne  system 
design  allow  very  low-level  flight  in  areas  where  that  is  possible. 

■  While  the  ground-based  system  detected  every  emplaced  ordnance  item 
within  its  actual  survey  area,  neither  helicopter-based  system  demonstrated 
reliable  detection  of  small  ordnance  (25%  probability  of  detection  or  less 
against  60  mm  mortar  rounds).  In  addition,  aMTADS  detected  only  three- 
fourths  of  the  105  mm  rounds  and  ORAGS  detected  less  than  one-third  of 
those  rounds.  Those  detection  statistics  would  argue  against  the  use  of  the 
helicopter  systems  for  individual  UXO  detection,  except  for  large  ordnance  in 
benign  topography,  land  cover,  and  magnetic  background  situations. 

■  aMTADS’s  consistent  ability  to  more  accurately  determine  target  position 
(33  cm  average  radial  error  on  the  emplaced  targets  vs.  97  cm  for  ORAGS) 
improves  target  reacquisition  performance  and  reduces  ambiguity 
opportunities. 

■  Current  automated  target  detection  and  recognition  algorithms  do  not  appear 
to  be  robust  or  accurate  enough  to  replace  the  use  of  human  judgment  in 
interpreting  the  magnetometer  data  and  determining  which  targets  are  likely 
UXO. 
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ACRONYMS 


aMTADS 

Airborne  Multi-Sensor  Towed  Array  Detection  System 

aMTADSE 

Airborne  Multi-Sensor  Towed  Array  Detection  System- 

Extended  Data  Set 

ERDC 

Engineering  R&D  Center 

ESTCP 

Environmental  Security  Technology  Certification  Program 

gMTADS 

Vehicle  Towed  Multi-Sensor  Towed  Array  Detection 
System 

GPS 

Global  Positioning  System 

IDA 

Institute  for  Defense  Analyses 

MTADS 

Multi-Sensor  Towed  Array  Detection  System 

NRL 

Naval  Research  Laboratory 

ORAGS 

Oak  Ridge  Airborne  Geophysics  System 

ORNL 

Oak  Ridge  National  Laboratory 

Pd 

probability  of  detection 

ROC 

receiver  operating  characteristics 

SERDP 

Strategic  Environmental  Research  and  Development 
Program 

UXO 

unexploded  ordnance 
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APPENDIX  A 

CHANCE  DETECTION  ANALYSIS 


Evaluators  long  ago  realized  that  in  analyzing  detection  perfonnance,  some 
number  of  detections  were  likely  to  be  due  to  chance.  That  is,  a  certain  number  of 
detections  that  are  graded  as  true  detections  are  not  actually  associated  with  the  item 
assumed  to  be  detected.  To  illustrate  the  point,  if  N  detection  calls  were  randomly 
distributed  over  a  test  plot  with  area  A,  a  demonstrator  might  expect,  just  based  on 
chance,  to  achieve  an  apparent  probability  of  detection  approximately  equal  to  NuRp/A, 
where  R/t  is  the  radius  of  the  halo  used.  A  more  sophisticated  analysis  is  developed  in 
Reference  3  to  assess  the  number  of  “chance”  detections.  The  assumption  made  is  that 
the  survey  instrument  has  inherent  random  errors  in  locating  targets  that  can  be  described 
by  zero-mean  and  equal  variance  Gaussian  distributions  in  the  east  and  north  directions. 
For  that  assumption,  we  would  expect  that  the  probability  density  function  for  the  radial 
error  would  be  Rayleigh  and  the  probability  of  detecting  an  object  within  a  radius  R/,  of 
the  declared  location  would  be  given  by  [4]: 


P(z  < 


=  1  -e 


V2 


where  a  is  the  location  error  standard  deviation  in  the  east  and  north  directions.  If  the 
halo  is  substantially  larger  than  the  error  standard  deviation,  then  the  probability  that  a 
detected  target  lies  inside  the  halo  is  essentially  unity.  For  example,  for  an  R/,/cr=  2,  the 
probability  is  94%,  and  for  f?/,/cr=  2.5  it  is  99%.  From  the  Isleta  emplaced  object 
detections,  the  average  of  the  east  and  north  error  standard  deviations  was  69  cm  for 
ORAGS  and  32  cm  for  aMTADS.  The  1.5  m  standard  halo  used  in  the  Isleta  analysis  is 
just  more  than  twice  the  ORAGS  average  standard  deviation,  and  so  approximating  the 
probability  as  unity,  while  not  as  accurate  as  might  be  desirable,  is  still  reasonable.  In 
many  cases,  however,  we  have  also  provided  results  for  a  2.0  m  halo  to  illustrate  the 
detection  difference  that  value  provides  for  the  ORAGS.  Applying  the  approximation  that 
the  probability  distribution  function  equals  unity  to  the  analysis  gives  the  number  of 
“true”  detections  as 

T  =  M -U  F -a 
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where 


T  =  number  of  true  matches 

U=  true  number  of  missed  detections  =  B  -  T 

B  =  number  of  items  buried 

F  =  true  number  of  false  detections  =  D  -  T 

D  =  number  of  target  declarations 

a=  (area  of  halo)/(area  of  site) 

M  =  Sum  of  true  +  lucky  matches 


This  results  in  a  quadratic  in  T  whose  solution  is 

fl  ' 
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Curves  of  true  detections  as  a  function  of  halo  should  increase  as  halo  size  increases  and 
reach  an  asymptotic  value  that  is  indicative  of  sensor  detection  performance.  For  a  sensor 
that  produces  small  position  estimation  errors,  the  curve  rises  rapidly  to  its  final  value, 
but  for  a  sensor  with  less  accurate  positioning,  the  curve  asymptotes  more  slowly.  When 
actual  data  are  plotted,  however,  curves  of  true  detections  reach  a  peak  value  and 
eventually  start  to  decrease  with  increasing  halo.  The  apparent  decrease  is  due  to  a 
violation  of  the  assumptions  under  which  the  above  equations  were  derived.  The  analysis 
assumes  that  declarations  are  widely  enough  separated  so  that  halos  never  overlap  and 
that  portions  of  halos  never  lie  outside  the  analysis  area.  At  some  point,  those 
assumptions  break  down  and  the  number  of  chance  detections  does  not  continue  to  rise 
proportionally  to  the  halo  area,  so  more  chance  detections  are  calculated  than  actually 
occur.  However,  until  the  assumptions  are  violated,  the  above  development  gives  a  good 
estimate  of  true  versus  chance  detections. 
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APPENDIX  B 

THREE-SYSTEM  AREA  DIG  RESULTS 


This  section  provides  the  results  of  the  recovery  operation  based  on  gMTADS 
category  1  and  2  calls  in  the  area  covered  by  all  three  systems.  The  coordinates  provided 
to  the  dig  team  were  based  on  gMTADS  positions;  however,  the  coordinates  in  the  table 
are  the  actual  locations  of  the  items  as  they  were  recovered.  IDA  sorted  the  items  into 
five  categories  for  scoring:  intact  ordnance,  ordnance  fragments,  non-ordnance-related 
clutter,  geology,  and  empty.  The  match  list  is  based  on  a  1.5  m  halo  around  the  recovered 
item,  and  the  miss  distance  for  each  system  is  provided. 
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Table  B-l.  Three-System  Area  Dig  Results 


Ground-Truth 

gMTADS 

aMTADSE 

ORAGS 

ID 

DESCRIPTION 

UTMNORTH 

UTMEAST 

ID 

UTMNORTH 

UTMEAST 

Call 

Miss 

(m) 

ID 

UTMNORTH 

UTMEAST 

Call 

Miss 

(m) 

ID 

UTMNORTH 

UTMEAST 

Call 

Miss 

(m) 

AS1-1 

MK-76  Practice  Bomb 

3857181.50 

318510.23 

17 

3857181.40 

318510.16 

1 

0.12 

612 

3857181.31 

318510.04 

1 

0.27 

AS  1-2 

MK-76  Practice  Bomb 

3857178.03 

318532.68 

18 

3857177.87 

318532.18 

1 

0.53 

610 

3857177.74 

318532.14 

1 

0.61 

AS  1-3 

M38  Bomb  Body  Fragments 

3857172.90 

318577.46 

AS  1-4 

MK-76  Practice  Bomb 

3857174.27 

318674.28 

31 

3857174.63 

318674.13 

1 

0.39 

21 

3857174.92 

318674.34 

2 

0.65 

AS  1-5 

M38  Bomb  Body  Fragments 

3857166.30 

318691.35 

34 

3857166.87 

318690.93 

1 

0.71 

25 

3857167.34 

318691.49 

4 

1.05 

AS1-6 

M38  Bomb  Body  Fragments 

3857185.84 

318715.57 

41 

3857185.47 

318715.22 

1 

0.51 

31 

3857185.36 

318715.09 

1 

0.68 

8679 

3857184.5 

318715.5 

6 

1.34 

AS  1-8 

M38  Bomb  Body  Fragments 

3857192.14 

318728.54 

50 

3857192.53 

318728.27 

1 

0.47 

33 

3857192.89 

318728.49 

1 

0.75 

16 

3857191.5 

318728.0 

1 

0.84 

AS  1-9 

M38  Bomb  Body  Fragments 

3857197.13 

318660.43 

63 

3857197.13 

318661.12 

1 

0.69 

24 

3857197.03 

318661.32 

3 

0.90 

AS1-10 

M38  Bomb  Body  Fragments 

3857193.70 

318666.17 

64 

3857194.28 

318665.07 

1 

1.24 

23 

3857194.46 

318665.35 

1 

1.12 

51 

3857194.0 

318665.5 

1 

0.73 

AS1-11 

M38  Bomb  Body  Fragments 

3857203.21 

318604.65 

68 

3857203.03 

318604.60 

1 

0.19 

17 

3857203.14 

318604.55 

1 

0.12 

5286 

3857202.5 

318604.5 

4 

0.73 

AS1-12 

M38  Bomb  Body  Fragments 

3857223.81 

318499.19 

78 

3857223.89 

318499.36 

1 

0.19 

616 

3857223.89 

318499.20 

2 

0.08 

5108 

3857224.5 

318499.5 

4 

0.76 

AS1-13 

M38  Bomb  Body  Fragments 

3857233.18 

318531.92 

85 

3857232.96 

318532.25 

1 

0.40 

619 

3857233.01 

318532.44 

1 

0.55 

4861 

3857233.0 

318532.5 

4 

0.61 

AS1-14 

MK-76  Practice  Bomb 

3857212.68 

318628.79 

96 

3857212.23 

318628.43 

1 

0.58 

42 

3857211.86 

318628.21 

2 

1.00 

AS1-15 

Magnetic  Rock 

3857224.24 

318639.74 

98 

3857224.12 

318639.75 

1 

0.12 

8094 

3857225.0 

318641.0 

5 

1.47 

AS1-16 

M38  Bomb  Body  Fragments 

3857221.95 

318696.09 

107 

3857221.75 

318696.28 

1 

0.28 

41 

3857221.82 

318695.91 

1 

0.22 

106 

3857221.5 

318697.0 

1 

1.02 

AS1-17 

M38  Bomb  Body  Fragments 

3857211.47 

318729.12 

116 

3857211.52 

318729.20 

1 

0.09 

39 

3857211.48 

318729.14 

1 

0.02 

185 

3857211.0 

318729.0 

2 

0.49 

AS1-18 

M38  Bomb  Body  Fragments 

3857239.87 

318698.84 

126 

3857239.72 

318699.10 

1 

0.30 

57 

3857239.45 

318698.91 

3 

0.43 

2794 

3857238.5 

318699.0 

3 

1.38 

AS1-19 

M38  Bomb  Body  Fragments 

3857246.04 

318674.57 

134 

3857246.14 

318674.70 

1 

0.16 

54 

3857245.95 

318674.76 

1 

0.21 

1138 

3857245.5 

318675.5 

3 

1.08 

AS  1-20 

M38  Bomb  Body  Fragments 

3857259.78 

318662.87 

140 

3857259.87 

318662.91 

1 

0.10 

53 

3857259.57 

318663.03 

1 

0.26 

AS1-21 

MK-76  Practice  Bomb 

3857245.07 

318583.92 

153 

3857244.90 

318583.95 

1 

0.17 

45 

3857244.93 

318583.96 

1 

0.15 

2700 

3857245.0 

318584.5 

3 

0.58 

AS  1-22 

M38  Bomb  Body  Fragments 

3857250.30 

318554.85 

157 

3857250.17 

318554.88 

1 

0.13 

620 

3857250.14 

318554.87 

1 

0.16 

116 

3857249.5 

318554.5 

1 

0.87 

AS  1-23 

M38  Bomb  Body  Fragments 

3857269.21 

318500.90 

166 

3857269.37 

318500.56 

1 

0.38 

631 

3857269.22 

318500.58 

1 

0.32 

300 

3857269.0 

318500.5 

2 

0.45 

AS  1-24 

MK-76  Practice  Bomb 

3857287.87 

318526.44 

183 

3857287.67 

318526.48 

1 

0.20 

639 

3857287.30 

318526.39 

1 

0.57 

AS  1-25 

M38  Bomb  Body  Fragments 

3857262.60 

318540.91 

187 

3857262.69 

318540.97 

1 

0.11 

622 

3857262.78 

318540.94 

1 

0.18 

932 

3857262.0 

318541.0 

2 

0.61 

AS  1-26 

MK-76  Practice  Bomb 

3857270.26 

318573.25 

189 

3857269.97 

318573.33 

1 

0.30 

78 

3857269.79 

318573.29 

1 

0.47 

1608 

3857270.5 

318574.0 

3 

0.79 

AS  1-27 

AN/M57  500#  Bomb 

3857266.08 

318620.35 

195 

3857265.85 

318620.14 

1 

0.31 

75 

3857265.97 

318620.14 

1 

0.24 

1105 

3857266.5 

318619.5 

3 

0.95 

AS  1-28 

AN/M57  500#  Bomb 

3857277.06 

318630.90 

196 

3857276.72 

318630.60 

1 

0.45 

74 

3857276.75 

318630.62 

1 

0.42 

AS  1-29 

M38  Bomb  Body  Fragments 

3857276.50 

318644.30 

199 

3857276.35 

318644.26 

1 

0.16 

71 

3857276.32 

318644.24 

1 

0.19 

AS1-30 

Tin  Can 

3857282.27 

318739.02 

219 

3857282.32 

318739.02 

1 

0.05 

5617 

3857283.5 

318739.0 

4 

1.23 

AS1-31 

M38  Bomb  Body  Fragments 

3857293.97 

318710.95 

232 

3857294.05 

318711.07 

1 

0.14 

64 

3857294.01 

318710.90 

1 

0.06 

130 

3857293.0 

318710.0 

1 

1.36 

AS  1-32 

M38  Bomb  Body  Fragments 

3857302.68 

318697.68 

236 

3857302.70 

318697.58 

1 

0.10 

65 

3857302.88 

318697.75 

1 

0.21 

117 

3857302.0 

318697.0 

1 

0.96 

AS1-33 

M38  Bomb  Body  Fragments 

3857303.61 

318688.06 

240 

3857303.40 

318687.96 

1 

0.23 

66 

3857303.65 

318688.04 

1 

0.04 

112 

3857303.5 

318689.0 

1 

0.95 

AS  1-34 

M38  Bomb  Body  Fragments 

3857309.53 

318698.40 

241 

3857309.32 

318698.30 

1 

0.23 

91 

3857308.90 

318698.17 

2 

0.67 

AS1-35 

M38  Bomb  Body  Fragments 

3857310.54 

318682.20 

242 

3857310.42 

318682.07 

1 

0.18 

88 

3857310.24 

318681.86 

2 

0.45 

AS1-36 

MK-76  Practice  Bomb 

3857300.75 

318555.07 

257 

3857300.49 

318554.98 

1 

0.28 

641 

3857300.57 

318555.07 

1 

0.18 

4487 

3857301.0 

318556.5 

4 

1.45 

AS1-37 

MK-76  Practice  Bomb 

3857287.58 

318517.01 

266 

3857286.87 

318516.70 

1 

0.77 

636 

3857286.81 

318517.39 

1 

0.86 

107 

3857287.0 

318516.0 

1 

1.16 

AS1-38 

MK-76  Practice  Bomb 

3857317.02 

318522.66 

274 

3857316.88 

318522.64 

1 

0.14 

650 

3857317.16 

318522.60 

1 

0.15 

AS1-39 

M38  Bomb  Body  Fragments 

3857326.86 

318552.10 

276 

3857326.77 

318552.38 

1 

0.29 

648 

3857326.91 

318552.22 

2 

0.13 

8435 

3857327.0 

318552.0 

5 

0.17 

AS  1-40 

Tin  Can  Fragments 

3857328.66 

318674.59 

285 

3857328.70 

318674.53 

1 

0.07 

7825 

3857329.5 

318674.5 

5 

0.84 

AS1-41 

M38  Bomb  Body  Fragments 

3857318.07 

318712.44 

296 

3857318.23 

318712.36 

1 

0.18 

95 

3857318.36 

318712.34 

1 

0.31 

AS  1-42 

M38  Bomb  Body  Fragments 

3857315.29 

318736.37 

306 

3857315.34 

318736.10 

1 

0.27 

99 

3857315.45 

318736.24 

1 

0.21 

786 

3857314.5 

318735.5 

2 

1.18 

AS  1-43 

Tin  Can 

3857329.61 

318749.69 

316 

3857329.75 

318749.75 

1 

0.15 

AS  1-44 

MK-76  Practice  Bomb 

3857350.22 

318562.60 

328 

3857350.17 

318562.64 

1 

0.06 

646 

3857350.20 

318562.52 

1 

0.08 

2691 

3857349.5 

318562.5 

3 

0.73 

AS  1-45 

M38  Bomb  Body  Fragments 

3857358.38 

318562.32 

332 

3857358.22 

318562.34 

1 

0.16 

660 

3857358.47 

318562.29 

1 

0.09 

AS  1-46 

MK-76  Practice  Bomb 

3857366.49 

318595.87 

340 

3857366.24 

318595.50 

1 

0.45 

120 

3857366.43 

318595.48 

1 

0.39 

796 

3857366.0 

318595.0 

2 

1.00 

AS  1-47 

Wire 

3857364.06 

318599.14 

341 

3857364.04 

318598.58 

1 

0.56 

AS  1-48 

MK-76  Practice  Bomb 

3857346.24 

318636.19 

347 

3857346.26 

318636.23 

1 

0.04 

116 

3857346.14 

318636.30 

1 

0.15 
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Ground-Truth 

gMTADS 

aMTADSE 

ORAGS 

ID 

DESCRIPTION 

UTMNORTH 

UTMEAST 

ID 

UTMNORTH 

UTMEAST 

Call 

Miss 

(m) 

ID 

UTMNORTH 

UTMEAST 

Call 

Miss 

(m) 

ID 

UTMNORTH 

UTMEAST 

Call 

Miss 

(m) 

AS  1-49 

Oil  Filter 

3857346.41 

318704.14 

369 

3857346.38 

318704.25 

1 

0.11 

7159 

3857346.0 

318704.0 

5 

0.43 

AS1-50 

M38  Bomb  Body  Fragments 

3857353.38 

318713.39 

372 

3857353.29 

318713.45 

1 

0.11 

109 

3857353.51 

318713.57 

2 

0.22 

AS1-51 

Pipe 

3857356.76 

318709.64 

373 

3857357.48 

318709.15 

1 

0.87 

110 

3857357.59 

318709.20 

1 

0.94 

AS  1-52 

MK-76  Practice  Bomb 

3857347.99 

318713.63 

375 

3857347.97 

318713.55 

1 

0.08 

107 

3857347.92 

318713.73 

1 

0.12 

710 

3857347.5 

318713.5 

2 

0.51 

AS1-53 

Tin  Cans 

3857341.15 

318745.95 

388 

3857341.46 

318745.57 

1 

0.49 

104 

3857341.36 

318745.71 

1 

0.32 

AS1-55 

M38  Bomb  Body  Fragments 

3857383.54 

318732.05 

392 

3857383.51 

318732.08 

1 

0.04 

9990 

3857382.5 

318732.0 

6 

1.04 

AS1-56 

M38  Bomb  Body  Fragments 

3857365.52 

318727.54 

397 

3857365.72 

318727.68 

1 

0.24 

108 

3857364.84 

318727.66 

3 

0.69 

2290 

3857364.5 

318727.0 

3 

1.15 

AS1-57 

M38  Bomb  Body  Fragments 

3857375.64 

318723.72 

398 

3857375.71 

318723.69 

1 

0.08 

161 

3857376.20 

318724.03 

2 

0.64 

5152 

3857377.0 

318723.5 

4 

1.38 

AS1-58 

M38  Bomb  Body  Fragments 

3857376.38 

318726.79 

399 

3857376.35 

318726.88 

1 

0.09 

861 

3857376.0 

318727.0 

2 

0.43 

AS1-59 

M38  Bomb  Body  Fragments 

3857384.66 

318724.17 

405 

3857384.56 

318724.20 

1 

0.10 

160 

3857384.71 

318724.00 

1 

0.18 

4956 

3857383.5 

318723.5 

4 

1.34 

AS1-60 

M38  Bomb  Body  Fragments 

3857374.78 

318657.98 

438 

3857374.90 

318657.79 

1 

0.22 

1046 

3857375.59 

318658.02 

5 

0.81 

2229 

3857374.0 

318658.0 

3 

0.78 

AS1-61 

M38  Bomb  Body  Fragments 

3857373.34 

318668.41 

439 

3857373.55 

318668.16 

1 

0.33 

147 

3857373.68 

318668.05 

3 

0.50 

627 

3857373.0 

318668.0 

2 

0.53 

AS  1-62 

M38  Bomb  Body  Fragments 

3857380.65 

318642.81 

445 

3857380.55 

318642.77 

1 

0.11 

139 

3857380.57 

318642.62 

1 

0.21 

2783 

3857379.5 

318642.5 

3 

1.19 

AS1-63 

M38  Bomb  Body  Fragments 

3857379.28 

318638.33 

446 

3857379.21 

318638.21 

1 

0.14 

138 

3857378.88 

318638.21 

2 

0.42 

2818 

3857379.0 

318637.5 

3 

0.88 

AS  1-64 

M38  Bomb  Body  Fragments 

3857384.47 

318627.42 

450 

3857384.55 

318627.46 

1 

0.09 

137 

3857384.65 

318627.83 

1 

0.45 

9306 

3857383.5 

318628.5 

6 

1.45 

AS1-65 

MK-76  Practice  Bomb 

3857386.37 

318605.62 

462 

3857386.69 

318605.51 

1 

0.34 

127 

3857386.98 

318605.36 

1 

0.66 

808 

3857385.5 

318605.5 

2 

0.88 

AS1-66 

MK-76  Practice  Bomb 

3857391.39 

318574.23 

467 

3857391.42 

318574.22 

1 

0.03 

1258 

3857391.67 

318574.26 

3 

0.28 

8599 

3857392.5 

318575.0 

6 

1.35 

AS1-67 

M38  Bomb  Body  Fragments 

3857385.82 

318569.52 

468 

3857385.70 

318569.14 

1 

0.40 

125 

3857385.59 

318568.88 

1 

0.68 

6967 

3857385.0 

318569.5 

5 

0.82 

AS1-68 

M38  Bomb  Body  Fragments 

3857377.86 

318551.48 

470 

3857377.73 

318551.14 

1 

0.36 

657 

3857377.72 

318551.29 

1 

0.24 

2628 

3857377.5 

318551.0 

3 

0.60 

AS1-69 

M38  Bomb  Body  Fragments 

3857367.89 

318557.55 

477 

3857368.14 

318557.27 

1 

0.38 

658 

3857368.51 

318557.26 

1 

0.68 

AS  1-70 

M38  Bomb  Body  Fragments 

3857369.06 

318562.67 

478 

3857368.89 

318562.50 

1 

0.24 

659 

3857368.72 

318562.67 

2 

0.34 

AS1-71 

M38  Bomb  Body  Fragments 

3857372.84 

318503.09 

488 

3857372.97 

318503.28 

1 

0.23 

654 

3857373.09 

318503.27 

1 

0.31 

515 

3857371.5 

318503.0 

2 

1.34 

AS  1-72 

M38  Bomb  Body  Fragments 

3857409.71 

318550.87 

510 

3857409.75 

318550.92 

1 

0.06 

677 

3857409.85 

318551.06 

2 

0.24 

7245 

3857409.5 

318551.0 

5 

0.25 

AS1-73 

Bomb,  M38  Practice 

3857411.78 

318581.06 

515 

3857411.46 

318581.37 

1 

0.45 

129 

3857411.42 

318581.34 

1 

0.46 

1117 

3857411.5 

318582.5 

3 

1.47 

AS  1-76 

MK-76  Practice  Bomb 

3857391.08 

318616.35 

518 

3857391.06 

318616.46 

1 

0.11 

135 

3857391.16 

318616.40 

1 

0.09 

822 

3857390.5 

318617.0 

2 

0.87 

AS  1-77 

M38  Bomb  Body  Fragments 

3857402.26 

318619.48 

519 

3857402.34 

318619.69 

1 

0.22 

136 

3857402.14 

318619.67 

1 

0.22 

1753 

3857402.5 

318620.0 

3 

0.57 

AS  1-78 

M38  Bomb  Body  Fragments 

3857400.28 

318652.20 

521 

3857400.38 

318651.91 

1 

0.31 

144 

3857400.18 

318652.13 

1 

0.12 

35 

3857400.0 

318653.0 

1 

0.85 

AS  1-79 

MK-76  Practice  Bomb 

3857408.30 

318637.85 

522 

3857408.69 

318637.66 

1 

0.43 

140 

3857408.67 

318637.95 

1 

0.38 

5695 

3857407.5 

318637.5 

4 

0.87 

AS1-80 

M38  Bomb  Body  Fragments 

3857414.46 

318642.85 

523 

3857414.38 

318642.72 

1 

0.15 

142 

3857414.31 

318642.73 

1 

0.19 

4936 

3857413.0 

318643.0 

4 

1.47 

AS1-81 

M38  Bomb  Body  Fragments 

3857411.12 

318659.21 

525 

3857411.16 

318659.43 

1 

0.22 

145 

3857411.15 

318659.47 

2 

0.26 

AS  1-82 

MK-76  Practice  Bomb 

3857415.75 

318660.67 

526 

3857415.87 

318660.75 

1 

0.14 

146 

3857416.13 

318661.10 

4 

0.57 

AS1-83 

M38  Bomb  Body  Fragments 

3857399.22 

318715.91 

534 

3857399.35 

318715.61 

1 

0.33 

159 

3857398.85 

318715.51 

2 

0.54 

1726 

3857399.0 

318716.5 

3 

0.63 

AS  1-84 

Bomb,  MK-83  Low  Drag 

3857411.32 

318714.23 

535 

3857410.93 

318714.04 

1 

0.43 

162 

3857411.22 

318714.12 

4 

0.15 

2932 

3857412.5 

318714.5 

4 

1.21 

AS1-87 

AN/M57  500#  Bomb 

3857412.95 

318751.61 

545 

3857412.68 

318751.19 

1 

0.50 

173 

3857412.79 

318750.98 

2 

0.65 

AS1-89 

M38  Bomb  Body  Fragments 

3857413.27 

318724.80 

551 

3857413.23 

318724.82 

1 

0.04 

165 

3857413.06 

318724.15 

3 

0.68 

AS1-90 

Bomb,  M38  Practice 

3857424.85 

318710.20 

553 

3857424.67 

318710.10 

1 

0.21 

163 

3857424.54 

318710.20 

1 

0.31 

AS1-91 

Bomb  Fragment 

3857423.95 

318702.64 

554 

3857423.98 

318702.68 

1 

0.05 

AS1-92 

MK-76  Practice  Bomb 

3857418.82 

318678.17 

557 

3857419.27 

318678.60 

1 

0.62 

152 

3857419.54 

318678.60 

2 

0.84 

81 

3857418.5 

318678.0 

1 

0.36 

AS1-93 

M38  Bomb  Body  Fragments 

3857422.30 

318680.59 

558 

3857422.05 

318681.00 

1 

0.48 

153 

3857421.99 

318681.22 

2 

0.70 

3337 

3857422.0 

318681.5 

4 

0.96 

AS1-95 

MK-76  Practice  Bomb 

3857430.03 

318593.64 

570 

3857430.01 

318593.95 

1 

0.31 

195 

3857429.90 

318593.95 

1 

0.34 

512 

3857429.5 

318593.5 

2 

0.55 

AS1-97 

M38  Bomb  Body  Fragments 

3857422.32 

318565.15 

572 

3857422.94 

318565.25 

1 

0.63 

206 

3857423.74 

318565.50 

3 

1.46 

AS1-98 

M38  Bomb  Body  Fragments 

3857413.76 

318562.53 

575 

3857413.63 

318562.21 

1 

0.35 

670 

3857413.66 

318562.68 

2 

0.18 

2523 

3857413.0 

318562.5 

3 

0.76 

AS1-99 

M38  Bomb  Body  Fragments 

3857426.54 

318553.23 

579 

3857426.00 

318552.92 

1 

0.62 

701 

3857426.63 

318552.81 

2 

0.43 

5562 

3857425.5 

318552.5 

4 

1.27 

AS1-100 

M38  Bomb  Body  Fragments 

3857427.05 

318522.52 

580 

3857427.06 

318522.56 

1 

0.04 

697 

3857427.04 

318522.44 

1 

0.08 

1855 

3857426.0 

318523.0 

3 

1.15 

AS1-101 

M38  Bomb  Body  Fragments 

3857417.83 

318515.70 

582 

3857417.59 

318515.57 

1 

0.27 

684 

3857417.52 

318515.66 

2 

0.31 

78 

3857417.5 

318516.0 

1 

0.45 

AS1-102 

M38  Bomb  Body  Fragments 

3857421.49 

318507.88 

583 

3857421.36 

318508.07 

1 

0.23 

AS1-105 

M38  Bomb  Body  Fragments 

3857433.29 

318503.25 

589 

3857433.13 

318503.37 

1 

0.20 

695 

3857432.90 

318503.05 

1 

0.44 

B-4 
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AS1-107 

AN/M57  500#  Bomb 

3857444.49 

318549.17 

596 

3857444.59 

318548.72 

1 

0.46 

700 

3857444.66 

318548.83 

1 

0.38 

15 

3857444.0 

318549.5 

1 

0.59 

AS1-108 

AN/M57  500#  Bomb 

3857465.48 

318542.12 

601 

3857465.35 

318542.08 

1 

0.14 

714 

3857465.25 

318541.91 

1 

0.31 

75 

3857464.5 

318541.5 

1 

1.16 

AS1-109 

MK-76  Practice  Bomb 

3857460.59 

318586.12 

612 

3857460.42 

318586.24 

1 

0.21 

205 

3857460.61 

318586.24 

2 

0.12 

4421 

3857459.5 

318585.5 

4 

1.25 

AS1-110 

MK-76  Practice  Bomb 

3857456.30 

318590.03 

614 

3857456.44 

318590.36 

1 

0.36 

204 

3857456.45 

318590.25 

1 

0.27 

837 

3857457.0 

318589.0 

2 

1.25 

AS1-111 

M38  Bomb  Body  Fragments 

3857455.93 

318595.10 

616 

3857455.89 

318595.35 

1 

0.25 

203 

3857456.10 

318595.34 

1 

0.29 

4929 

3857455.5 

318594.0 

4 

1.18 

AS1-112 

Bomb,  M38  Practice 

3857449.95 

318606.23 

621 

3857449.72 

318606.25 

1 

0.23 

198 

3857449.60 

318606.17 

1 

0.36 

111 

3857449.5 

318607.0 

1 

0.89 

AS1-113 

M38  Bomb  Body  Fragments 

3857446.60 

318614.79 

622 

3857446.50 

318614.74 

1 

0.11 

194 

3857445.98 

318614.30 

2 

0.79 

2680 

3857446.0 

318614.0 

3 

0.99 

AS1-114 

M38  Bomb  Body  Fragments 

3857456.80 

318610.43 

624 

3857456.96 

318610.54 

1 

0.19 

201 

3857457.01 

318610.66 

1 

0.31 

2716 

3857456.5 

318611.0 

3 

0.64 

AS1-115 

M38  Bomb  Body  Fragments 

3857462.36 

318605.19 

626 

3857461.92 

318605.29 

1 

0.45 

202 

3857461.89 

318605.25 

1 

0.47 

812 

3857461.0 

318605.5 

2 

1.39 

AS1-116 

M38  Bomb  Body  Fragments 

3857450.76 

318631.41 

628 

3857450.56 

318631.48 

1 

0.21 

193 

3857450.97 

318631.15 

2 

0.33 

2405 

3857449.5 

318631.0 

3 

1.33 

AS1-117 

M38  Bomb  Body  Fragments 

3857453.23 

318635.97 

629 

3857453.19 

318635.93 

1 

0.06 

1049 

3857453.66 

318635.96 

1 

0.43 

AS1-118 

M38  Bomb  Body  Fragments 

3857438.28 

318649.53 

630 

3857438.30 

318649.49 

1 

0.04 

189 

3857438.30 

318649.54 

2 

0.02 

662 

3857438.0 

318649.0 

2 

0.60 

AS1-119 

M38  Bomb  Body  Fragments 

3857460.00 

318643.35 

632 

3857459.97 

318643.33 

1 

0.04 

483 

3857459.91 

318643.55 

2 

0.22 

AS1-120 

MK-76  Practice  Bomb 

3857466.47 

318644.69 

633 

3857466.93 

318644.43 

1 

0.53 

485 

3857466.33 

318644.19 

2 

0.52 

5203 

3857465.5 

318644.5 

4 

0.99 

AS1-121 

M38  Bomb  Body  Fragments 

3857459.01 

318684.55 

639 

3857458.96 

318684.54 

1 

0.05 

495 

3857459.08 

318684.55 

3 

0.07 

AS1-122 

Bomb,  M38  Practice 

3857462.89 

318686.38 

640 

3857462.95 

318686.83 

1 

0.45 

496 

3857462.85 

318686.69 

1 

0.31 

1301 

3857462.5 

318686.5 

3 

0.41 

AS1-125 

M38  Bomb  Body  Fragments 

3857451.58 

318736.62 

647 

3857451.61 

318736.31 

1 

0.31 

AS1-126 

M38  Bomb  Body  Fragments 

3857470.18 

318508.20 

651 

3857469.82 

318508.25 

1 

0.36 

724 

3857469.87 

318508.28 

1 

0.32 

167 

3857469.0 

318508.5 

2 

1.22 

AS1-127 

Buster  Cup,  M38  Bomb 

3857466.07 

318506.59 

652 

3857466.16 

318506.51 

1 

0.12 

AS1-128 

M38  Bomb  Body  Fragments 

3857474.83 

318516.62 

654 

3857474.65 

318516.35 

1 

0.32 

726 

3857474.52 

318516.43 

2 

0.36 

AS1-129 

M38  Bomb  Body  Fragments 

3857481.62 

318508.03 

655 

3857481.62 

318508.14 

1 

0.11 

723 

3857481.79 

318508.04 

1 

0.17 

971 

3857481.0 

318508.5 

3 

0.78 

AS1-130 

Bomb,  Mk-81  Low  Drag 
Practice 

3857477.60 

318557.93 

670 

3857477.40 

318558.05 

1 

0.23 

710 

3857477.66 

318558.15 

1 

0.23 

AS1-131 

M38  Bomb  Body  Fragments 

3857475.15 

318584.25 

683 

3857475.10 

318584.27 

1 

0.05 

231 

3857475.22 

318584.07 

2 

0.19 

1929 

3857475.0 

318584.0 

3 

0.29 

AS1-132 

MK-76  Practice  Bomb 

3857483.55 

318585.47 

684 

3857483.34 

318585.79 

1 

0.38 

227 

3857483.44 

318585.89 

1 

0.43 

AS1-133 

M38  Bomb  Body  Fragments 

3857477.87 

318592.71 

685 

3857478.02 

318592.63 

1 

0.17 

229 

3857477.89 

318592.61 

3 

0.10 

74 

3857478.0 

318592.5 

1 

0.25 

AS1-134 

M38  Bomb  Body  Fragments 

3857474.25 

318595.06 

691 

3857474.28 

318595.34 

1 

0.28 

228 

3857474.39 

318595.35 

1 

0.32 

513 

3857474.0 

318595.0 

2 

0.26 

AS1-135 

M38  Bomb  Body  Fragments 

3857478.81 

318611.56 

693 

3857478.83 

318611.35 

1 

0.21 

214 

3857478.75 

318611.45 

1 

0.13 

5040 

3857478.0 

318612.0 

4 

0.92 

AS1-137 

M38  Bomb  Body  Fragments 

3857479.69 

318620.01 

697 

3857479.56 

318619.87 

1 

0.19 

481 

3857479.51 

318620.22 

2 

0.28 

917 

3857480.0 

318619.0 

2 

1.06 

AS1-138 

M38  Bomb  Body  Fragments 

3857483.14 

318630.62 

699 

3857483.25 

318630.68 

1 

0.13 

489 

3857483.12 

318630.58 

1 

0.04 

4692 

3857482.5 

318630.0 

4 

0.89 

AS1-139 

M38  Bomb  Body  Fragments 

3857490.72 

318641.03 

702 

3857490.45 

318640.98 

1 

0.27 

491 

3857490.19 

318640.91 

3 

0.54 

5247 

3857489.5 

318641.5 

4 

1.31 

AS1-141 

MK-76  Practice  Bomb 

3857472.25 

318672.60 

709 

3857472.07 

318672.09 

1 

0.54 

494 

3857472.68 

318672.36 

1 

0.49 

649 

3857472.0 

318673.0 

2 

0.47 

AS1-143 

M38  Bomb  Body  Fragments 

3857477.89 

318745.37 

725 

3857477.80 

318745.48 

1 

0.14 

508 

3857477.47 

318744.35 

2 

1.10 

AS1-144 

M38  Bomb  Body  Fragments 

3857478.90 

318748.41 

726 

3857478.87 

318748.30 

1 

0.11 

507 

3857478.89 

318748.16 

2 

0.25 

821 

3857486.0 

318524.5 

2 

1.08 

AS1-144 

MK-76  Practice  Bomb 

3857486.70 

318525.32 

754 

3857487.18 

318525.16 

1 

0.51 

718 

3857487.19 

318525.11 

1 

0.53 

657 

3857478.5 

318749.0 

2 

0.71 

AS1-145 

M38  Bomb  Body  Fragments 

3857477.50 

318757.13 

729 

3857477.56 

318757.23 

1 

0.12 

506 

3857477.70 

318757.00 

2 

0.24 

AS1-148 

M38  Bomb  Body  Fragments 

3857509.74 

318506.24 

741 

3857510.11 

318506.60 

1 

0.52 

730 

3857510.05 

318506.34 

1 

0.33 

864 

3857509.0 

318507.5 

2 

1.46 

AS1-149 

M38  Bomb  Body  Fragments 

3857510.16 

318522.03 

744 

3857509.98 

318522.23 

1 

0.27 

738 

3857509.69 

318521.97 

2 

0.47 

1468 

3857509.5 

318522.0 

3 

0.66 

AS1-150 

M38  Bomb  Body  Fragments 

3857505.91 

318527.17 

746 

3857506.00 

318527.22 

1 

0.10 

737 

3857506.05 

318527.23 

1 

0.15 

1491 

3857505.0 

318527.0 

3 

0.93 

AS1-151 

M38  Bomb  Body  Fragments 

3857493.78 

318519.35 

748 

3857493.99 

318519.38 

1 

0.21 

734 

3857494.59 

318519.19 

5 

0.83 

1670 

3857495.0 

318520.0 

3 

1.38 

AS1-152 

M38  Bomb  Body  Fragments 

3857498.09 

318523.46 

750 

3857498.67 

318524.05 

1 

0.83 

735 

3857498.29 

318523.87 

2 

0.46 

2131 

3857498.0 

318523.0 

3 

0.47 

AS1-153 

M38  Bomb  Body  Fragments 

3857491.36 

318521.04 

751 

3857490.76 

318521.12 

1 

0.61 

733 

3857490.89 

318521.82 

2 

0.91 

AS1-155 

M38  Bomb  Body  Fragments 

3857505.20 

318549.54 

761 

3857505.08 

318549.56 

1 

0.12 

744 

3857505.06 

318549.73 

1 

0.24 

728 

3857505.0 

318550.5 

2 

0.98 

AS1-156 

M38  Bomb  Body  Fragments 

3857491.79 

318560.53 

770 

3857491.74 

318560.45 

1 

0.09 

705 

3857491.57 

318560.40 

3 

0.26 

AS1-157 

M38  Bomb  Body  Fragments 

3857501.51 

318559.02 

772 

3857501.69 

318558.81 

1 

0.28 

742 

3857501.83 

318558.86 

1 

0.36 

2599 

3857500.5 

318559.5 

3 

1.12 

AS1-158 

M38  Bomb  Body  Fragments 

3857506.04 

318562.43 

773 

3857506.18 

318562.46 

1 

0.14 

743 

3857506.25 

318562.36 

1 

0.22 

54 

3857506.0 

318563.0 

1 

0.57 

AS1-159 

M38  Bomb  Body  Fragments 

3857497.97 

318567.64 

774 

3857498.38 

318567.37 

1 

0.49 

212 

3857498.40 

318567.60 

3 

0.43 

AS1-160 

Bomb,  AN/M64  1000# 

3857505.82 

318582.41 

778 

3857505.70 

318582.61 

1 

0.23 

220 

3857505.76 

318582.47 

1 

0.08 

1416 

3857506.0 

318583.5 

3 

1.10 

B-5 
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gMTADS 

aMTADSE 

ORAGS 

ID 

DESCRIPTION 

UTMNORTH 

UTMEAST 

ID 

UTMNORTH 

UTMEAST 

Call 

Miss 

(m) 

ID 

UTMNORTH 

UTMEAST 

Call 

Miss 

(m) 

ID 

UTMNORTH 

UTMEAST 

Call 

Miss 

(m) 

AS1-161 

M38  Bomb  Body  Fragments 

3857491.90 

318609.76 

787 

3857491.94 

318609.72 

1 

0.06 

216 

3857492.05 

318610.02 

2 

0.30 

AS1-163 

M38  Bomb  Body  Fragments 

3857506.83 

318650.86 

795 

3857506.82 

318650.69 

1 

0.17 

4906 

3857507.5 

318651.5 

4 

0.93 

AS1-164 

M38  Bomb  Body  Fragments 

3857506.98 

318658.02 

800 

3857506.95 

318658.00 

1 

0.04 

538 

3857506.19 

318657.90 

3 

0.80 

10 

3857506.5 

318658.5 

1 

0.68 

AS1-173 

M38  Bomb  Body  Fragments 

3857528.32 

318752.24 

846 

3857528.40 

318751.75 

1 

0.50 

514 

3857528.42 

318751.74 

3 

0.51 

AS1-178 

MK-76  Practice  Bomb 

3857519.72 

318674.72 

883 

3857520.21 

318674.40 

1 

0.59 

535 

3857520.30 

318674.43 

1 

0.65 

595 

3857519.0 

318673.5 

2 

1.42 

AS1-180 

M38  Bomb  Body  Fragments 

3857514.73 

318636.02 

891 

3857514.73 

318636.07 

1 

0.05 

543 

3857514.31 

318636.48 

2 

0.62 

446 

3857514.5 

318635.5 

2 

0.57 

AS1-184 

MK-76  Practice  Bomb 

3857525.84 

318584.75 

911 

3857525.33 

318584.41 

1 

0.61 

546 

3857525.05 

318584.33 

1 

0.89 

AS1-185 

M38  Bomb  Body  Fragments 

3857398.93 

318735.67 

540 

3857399.04 

318735.48 

1 

0.22 

167 

3857399.10 

318735.63 

1 

0.17 

1052 

3857398.0 

318735.5 

3 

0.95 

AS1-187 

MK-76  Practice  Bomb 

3857533.17 

318575.28 

917 

3857532.85 

318575.58 

1 

0.44 

548 

3857533.06 

318575.67 

1 

0.41 

104 

3857533.0 

318575.0 

1 

0.33 

AS1-188 

MK-76  Practice  Bomb 

3857539.64 

318584.29 

918 

3857538.99 

318584.27 

1 

0.65 

549 

3857539.24 

318584.42 

1 

0.42 

83 

3857538.5 

318584.0 

1 

1.18 

AS1-189 

M38  Bomb  Body  Fragments 

3857529.66 

318566.78 

921 

3857529.48 

318566.62 

1 

0.24 

752 

3857529.34 

318566.73 

1 

0.32 

2202 

3857529.0 

318566.5 

3 

0.72 

AS1-190 

MK-76  Practice  Bomb 

3857542.76 

318571.62 

923 

3857542.31 

318570.71 

1 

1.02 

1055 

3857542.49 

318570.57 

2 

1.08 

9215 

3857542.0 

318571.5 

6 

0.77 

AS1-191 

M38  Bomb  Body  Fragments 

3857523.39 

318558.90 

936 

3857523.25 

318559.02 

1 

0.18 

753 

3857523.30 

318558.86 

1 

0.10 

4623 

3857522.5 

318559.5 

4 

1.07 

AS1-192 

M38  Bomb  Body  Fragments 

3857520.31 

318556.36 

937 

3857520.34 

318556.30 

1 

0.07 

758 

3857520.42 

318556.23 

1 

0.17 

96 

3857520.5 

318556.0 

1 

0.41 

AS1-193 

M38  Bomb  Body  Fragments 

3857516.78 

318564.73 

938 

3857516.96 

318564.92 

1 

0.26 

754 

3857517.25 

318564.76 

1 

0.47 

788 

3857516.5 

318564.5 

2 

0.36 

AS1-194 

M38  Bomb  Body  Fragments 

3857514.48 

318549.89 

942 

3857514.35 

318549.92 

1 

0.13 

759 

3857514.12 

318549.94 

1 

0.36 

110 

3857514.0 

318550.0 

1 

0.49 

AS1-195 

M38  Bomb  Body  Fragments 

3857521.26 

318549.70 

944 

3857521.27 

318549.83 

1 

0.13 

770 

3857521.41 

318549.40 

1 

0.34 

4126 

3857521.0 

318551.0 

4 

1.33 

AS1-196 

M38  Bomb  Body  Fragments 

3857531.98 

318540.47 

955 

3857532.13 

318540.11 

1 

0.39 

774 

3857531.90 

318540.00 

1 

0.48 

AS1-197 

M38  Bomb  Body  Fragments 

3857535.33 

318534.78 

959 

3857535.33 

318534.80 

1 

0.02 

778 

3857535.30 

318535.14 

3 

0.36 

4913 

3857534.5 

318535.0 

4 

0.86 

AS1-198 

M38  Bomb  Body  Fragments 

3857535.33 

318534.78 

AS1-199 

M38  Bomb  Body  Fragments 

3857539.04 

318523.93 

962 

3857539.08 

318523.98 

1 

0.06 

785 

3857539.24 

318524.09 

1 

0.26 

7008 

3857538.5 

318524.0 

5 

0.54 

AS1-200 

M38  Bomb  Body  Fragments 

3857525.35 

318522.07 

968 

3857525.18 

318522.28 

1 

0.27 

787 

3857525.18 

318522.40 

3 

0.37 

AS  1-201 

M38  Bomb  Body  Fragments 

3857521.46 

318517.54 

970 

3857521.54 

318517.53 

1 

0.08 

788 

3857521.51 

318517.56 

1 

0.05 

AS  1-202 

M38  Bomb  Body  Fragments 

3857530.56 

318512.40 

972 

3857530.42 

318512.26 

1 

0.20 

793 

3857530.50 

318512.26 

2 

0.15 

1779 

3857529.5 

318512.0 

3 

1.13 

AS1-203 

M38  Bomb  Body  Fragments 

3857535.28 

318508.83 

976 

3857535.36 

318508.83 

1 

0.08 

AS1-207 

M38  Bomb  Body  Fragments 

3857519.78 

318507.28 

990 

3857519.65 

318507.13 

1 

0.20 

AS1-211 

M38  Bomb  Body  Fragments 

3857549.65 

318508.45 

1000 

3857549.31 

318508.52 

1 

0.35 

849 

3857549.35 

318508.81 

1 

0.47 

6010 

3857548.5 

318509.0 

5 

1.27 

AS1-212 

M38  Bomb  Body  Fragments 

3857550.13 

318510.66 

1001 

3857550.48 

318510.60 

1 

0.36 

AS1-213 

M38  Bomb  Body  Fragments 

3857552.16 

318512.48 

1002 

3857552.16 

318512.13 

1 

0.35 

850 

3857551.02 

318512.67 

3 

1.16 

AS1-214 

M38  Bomb  Body  Fragments 

3857542.46 

318506.93 

AS1-215 

CLAMP,  Missile  Warhead 

3857541.01 

318514.95 

3031 

3857540.5 

318516.0 

4 

1.17 

AS1-216 

M38  Bomb  Body  Fragments 

3857548.13 

318512.89 

1012 

3857548.12 

318512.91 

1 

0.02 

AS1-217 

M38  Bomb  Body  Fragments 

3857545.30 

318519.12 

1013 

3857545.19 

318519.30 

1 

0.21 

790 

3857545.13 

318519.37 

1 

0.30 

1232 

3857544.5 

318519.5 

3 

0.89 

AS1-218 

M38  Bomb  Body  Fragments 

3857549.94 

318521.36 

1017 

3857550.40 

318521.31 

1 

0.46 

AS1-219 

M38  Bomb  Body  Fragments 

3857551.24 

318525.47 

1018 

3857551.68 

318525.40 

1 

0.45 

781 

3857550.97 

318525.66 

2 

0.33 

79 

3857551.5 

318526.0 

1 

0.59 

AS  1-220 

M38  Bomb  Body  Fragments 

3857553.04 

318527.20 

1019 

3857552.89 

318526.97 

1 

0.27 

AS  1-221 

M38  Bomb  Body  Fragments 

3857543.76 

318532.53 

1021 

3857544.16 

318532.14 

1 

0.56 

AS  1-222 

M38  Bomb  Body  Fragments 

3857546.09 

318532.79 

1022 

3857545.95 

318532.52 

1 

0.30 

291 

3857545.0 

318532.5 

2 

1.13 

AS  1-223 

M38  Bomb  Body  Fragments 

3857544.72 

318535.81 

1023 

3857544.47 

318535.50 

1 

0.40 

777 

3857544.83 

318535.12 

2 

0.70 

600 

3857544.5 

318535.5 

2 

0.38 

AS  1-224 

M38  Bomb  Body  Fragments 

3857552.69 

318536.33 

1027 

3857552.48 

318536.22 

1 

0.24 

784 

3857552.23 

318536.68 

3 

0.58 

AS  1-225 

M38  Bomb  Body  Fragments 

3857541.00 

318545.97 

1030 

3857541.13 

318545.97 

1 

0.13 

768 

3857541.47 

318546.56 

1 

0.75 

AS  1-226 

M38  Bomb  Body  Fragments 

3857541.62 

318548.69 

1031 

3857541.75 

318548.56 

1 

0.18 

767 

3857541.80 

318548.61 

1 

0.20 

499 

3857541.0 

318548.5 

2 

0.65 

AS  1-227 

M38  Bomb  Body  Fragments 

3857543.87 

318553.28 

1036 

3857544.00 

318553.15 

1 

0.18 

763 

3857543.67 

318552.36 

1 

0.94 

3668 

3857544.5 

318552.5 

4 

1.00 

AS  1-228 

M38  Bomb  Body  Fragments 

3857548.84 

318562.18 

1040 

3857548.72 

318561.84 

1 

0.36 

762 

3857548.72 

318562.11 

1 

0.14 

AS  1-229 

M38  Bomb  Body  Fragments 

3857550.13 

318555.16 

1041 

3857550.93 

318555.13 

1 

0.80 

803 

3857550.93 

318555.18 

1 

0.80 

565 

3857550.5 

318554.0 

2 

1.22 

AS1-230 

M38  Bomb  Body  Fragments 

3857554.33 

318552.48 

1042 

3857554.26 

318552.51 

1 

0.08 

804 

3857554.20 

318552.68 

1 

0.24 

3417 

3857553.5 

318553.5 

4 

1.32 

AS  1-231 

M38  Bomb  Body  Fragments 

3857557.12 

318561.33 

1043 

3857557.21 

318561.24 

1 

0.13 

810 

3857557.08 

318561.14 

2 

0.19 

842 

3857556.0 

318560.5 

2 

1.39 

B-6 
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aMTADSE 

ORAGS 

ID 
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UTMEAST 

ID 
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ID 
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ID 
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AS  1-232 

M38  Bomb  Body  Fragments 

3857563.02 

318554.62 

1046 

3857563.20 

318554.68 

1 

0.19 

821 

3857563.25 

318554.71 

3 

0.25 

AS1-233 

M38  Bomb  Body  Fragments 

3857559.68 

318552.64 

1047 

3857559.71 

318552.58 

1 

0.07 

823 

3857559.55 

318552.44 

1 

0.24 

1635 

3857559.5 

318552.5 

3 

0.23 

AS  1-234 

M38  Bomb  Body  Fragments 

3857566.10 

318550.63 

1048 

3857566.09 

318550.65 

1 

0.02 

822 

3857565.74 

318550.83 

1 

0.41 

AS1-235 

M38  Bomb  Body  Fragments 

3857570.11 

318553.67 

1049 

3857570.11 

318553.82 

1 

0.15 

820 

3857569.92 

318553.91 

1 

0.31 

3920 

3857571.0 

318553.0 

4 

1.11 

AS1-236 

M38  Bomb  Body  Fragments 

3857560.55 

318548.13 

1050 

3857560.64 

318547.72 

1 

0.42 

824 

3857560.84 

318547.53 

1 

0.67 

716 

3857560.0 

318547.5 

2 

0.84 

AS1-237 

M38  Bomb  Body  Fragments 

3857563.46 

318539.63 

1058 

3857562.95 

318539.55 

1 

0.52 

AS1-238 

M38  Bomb  Body  Fragments 

3857569.32 

318538.96 

1059 

3857569.38 

318538.88 

1 

0.10 

830 

3857569.34 

318538.86 

1 

0.10 

1793 

3857568.5 

318539.0 

3 

0.82 

AS1-239 

M38  Bomb  Body  Fragments 

3857548.43 

318570.17 

1060 

3857548.52 

318569.81 

1 

0.37 

608 

3857548.62 

318569.87 

2 

0.36 

4958 

3857548.0 

318569.5 

4 

0.80 

AS  1-240 

M38  Bomb  Body  Fragments 

3857569.42 

318571.81 

1066 

3857569.17 

318571.83 

1 

0.25 

605 

3857569.06 

318572.04 

1 

0.43 

2335 

3857568.5 

318571.5 

3 

0.97 

AS  1-241 

M38  Bomb  Body  Fragments 

3857549.82 

318578.28 

1071 

3857549.87 

318578.03 

1 

0.26 

606 

3857550.16 

318578.02 

3 

0.43 

7334 

3857550.0 

318578.0 

5 

0.33 

AS  1-243 

M38  Bomb  Body  Fragments 

3857550.77 

318586.96 

1078 

3857550.66 

318587.16 

1 

0.23 

555 

3857550.60 

318586.98 

2 

0.17 

AS  1-246 

M38  Bomb  Body  Fragments 

3857555.32 

318602.13 

1083 

3857555.70 

318601.97 

1 

0.41 

562 

3857555.51 

318601.50 

3 

0.66 

AS  1-247 

M38  Bomb  Body  Fragments 

3857547.45 

318613.69 

1086 

3857547.52 

318613.72 

1 

0.08 

552 

3857546.98 

318613.64 

3 

0.47 

650 

3857547.0 

318613.0 

2 

0.82 

AS  1-248 

MK-76  Practice  Bomb 

3857553.24 

318610.96 

1089 

3857553.62 

318610.72 

1 

0.45 

553 

3857553.71 

318610.53 

1 

0.64 

AS  1-249 

MK-76  Practice  Bomb 

3857557.43 

318604.62 

1090 

3857557.74 

318604.32 

1 

0.43 

563 

3857557.99 

318604.25 

1 

0.67 

6491 

3857557.5 

318605.0 

5 

0.39 

AS1-250 

M38  Bomb  Body  Fragments 

3857559.90 

318609.63 

1091 

3857559.92 

318609.75 

1 

0.12 

1063 

3857559.51 

318609.95 

3 

0.50 

6683 

3857560.5 

318611.0 

5 

1.50 

AS  1-252 

M38  Bomb  Body  Fragments 

3857564.39 

318619.96 

1095 

3857564.49 

318620.07 

1 

0.15 

1062 

3857564.49 

318620.18 

1 

0.24 

AS1-253 

M38  Bomb  Body  Fragments 

3857557.18 

318632.99 

1101 

3857557.03 

318633.19 

1 

0.25 

1064 

3857557.28 

318633.33 

2 

0.35 

AS  1-254 

MK-76  Practice  Bomb 

3857551.95 

318634.75 

1102 

3857551.73 

318634.74 

1 

0.22 

1065 

3857552.14 

318634.83 

1 

0.21 

3937 

3857550.5 

318635.0 

4 

1.47 

AS1-256 

M38  Bomb  Body  Fragments 

3857565.93 

318665.03 

1123 

3857565.91 

318664.99 

1 

0.04 

575 

3857565.79 

318665.50 

2 

0.49 

2431 

3857565.0 

318664.0 

3 

1.39 

AS1-258 

MK-76  Practice  Bomb 

3857559.48 

318679.45 

1131 

3857559.41 

318678.86 

1 

0.59 

602 

3857559.48 

318678.59 

3 

0.86 

AS1-259 

M38  Bomb  Body  Fragments 

3857561.93 

318681.23 

1132 

3857562.08 

318680.73 

1 

0.52 

601 

3857562.15 

318680.79 

2 

0.49 

511 

3857561.5 

318680.0 

2 

1.30 

AS  1-262 

M38  Bomb  Body  Fragments 

3857562.08 

318697.43 

1140 

3857562.16 

318697.13 

1 

0.31 

603 

3857561.96 

318697.03 

3 

0.42 

2174 

3857562.0 

318697.0 

3 

0.44 

AS  1-264 

MK-76  Practice  Bomb 

3857554.67 

318734.59 

1147 

3857554.97 

318734.49 

1 

0.32 

592 

3857555.51 

318734.65 

2 

0.84 

66 

3857554.0 

318734.5 

1 

0.68 

AS1-265 

MK-76  Practice  Bomb 

3857559.27 

318739.75 

1156 

3857559.37 

318740.13 

1 

0.39 

591 

3857559.65 

318740.23 

2 

0.61 

69 

3857559.0 

318739.5 

1 

0.37 

AS1-268 

M38  Bomb  Body  Fragments 

3857565.37 

318752.71 

1166 

3857565.17 

318752.61 

1 

0.22 

587 

3857565.18 

318752.76 

1 

0.20 

921 

3857564.0 

318752.5 

2 

1.39 

AS1-269 

M38  Bomb  Body  Fragments 

3857567.14 

318748.65 

1169 

3857567.54 

318748.62 

1 

0.40 

589 

3857567.45 

318748.43 

3 

0.38 

4842 

3857566.0 

318748.5 

4 

1.15 

AS  1-271 

M38  Bomb  Body  Fragments 

3857583.29 

318755.09 

1176 

3857583.48 

318755.22 

1 

0.23 

596 

3857583.52 

318755.41 

1 

0.39 

2367 

3857582.5 

318755.0 

3 

0.80 

AS  1-274 

Possible  Nuclear  Simulator 

3857567.78 

318723.88 

1185 

3857568.62 

318723.43 

1 

0.95 

581 

3857568.60 

318723.43 

1 

0.94 

981 

3857567.5 

318724.5 

3 

0.68 

AS  1-276 

M38  Bomb  Body  Fragments 

3857574.66 

318616.91 

1215 

3857574.53 

318617.31 

1 

0.42 

566 

3857574.18 

318617.36 

2 

0.66 

4147 

3857574.0 

318617.0 

4 

0.67 

AS1-277 

M38  Bomb  Body  Fragments 

3857573.77 

318598.15 

1223 

3857573.77 

318598.00 

1 

0.15 

561 

3857573.56 

318597.94 

1 

0.30 

AS  1-278 

M38  Bomb  Body  Fragments 

3857581.06 

318598.31 

1224 

3857581.08 

318598.47 

1 

0.16 

AS1-280 

M38  Bomb  Body  Fragments 

3857585.20 

318589.31 

1229 

3857585.52 

318589.81 

1 

0.59 

AS  1-281 

M38  Bomb  Body  Fragments 

3857584.93 

318602.00 

1230 

3857585.18 

318601.73 

1 

0.37 

1150 

3857585.73 

318601.88 

1 

0.81 

527 

3857585.0 

318601.0 

2 

1.00 

AS1-282 

MK-76  Practice  Bomb 

3858655.25 

318729.14 

1250 

3858654.97 

318729.53 

1 

0.48 

277 

3858654.39 

318728.94 

2 

0.88 

236 

3858654.0 

318729.0 

2 

1.26 

AS  1-294 

MK-76  Practice  Bomb 

3858721.80 

318653.55 

1293 

3858721.98 

318653.61 

1 

0.19 

250 
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APPENDIX  C 

DISCRIMINATION  RESULTS 


This  appendix  provides  discrimination  results  based  on  the  gMTADS  category  1 
(most  likely  UXO)  items  dug  from  the  three  system  area.  Two  tables  are  provided  that 
cross-reference  the  aMTADS  and  ORAGS  picks.  Table  C-l  sorts  by  ORAGS  picks,  and 
then  within  each  category  the  corresponding  aMTADS  picks  are  sorted.  Table  C-2 
reverses  the  procedure,  sorting  first  by  aMTADS  picks  and  then  sorting  the  ORAGS 
picks  within  each  of  those  categories. 

The  first  thing  to  note  from  the  top  row  of  data  in  each  table  is  that  the  gMTADS 
category  1  selections  did  a  very  good  job  of  discriminating  ordnance-related  items  from 
other  objects  (only  8  items  of  221  were  not  ordnance-related).  However,  a  much  poorer 
job  was  done  of  discriminating  intact  ordnance  from  ordnance  fragments,  with  about 
three  times  as  many  fragments  being  selected  as  intact  ordnance  items.  Because  only  39 
gMTADS  category  2  items  were  dug,  complete  tables  for  those  items  are  not  provided. 
Of  the  category  2  items,  6  were  intact  ordnance,  27  were  ordnance  fragments,  4  were 
non-ordnance  clutter,  and  2  were  geologic  in  origin.  Thus,  results  were  similar  to  those 
for  category  1  items,  where  a  good  job  was  done  in  discriminating  ordnance-related  items 
from  others,  but  not  intact  ordnance  from  UXO  fragments. 

From  Table  C-2  we  see  that  aMTADS  matched  39  of  the  intact  ordnance  items  in 
category  1  and  10  additional  items  in  category  2.  Two  each  were  listed  as  categories  3 
and  4.  One  intact  ordnance  item  was  not  detected.  Further,  115  of  the  159  ordnance 
fragment  items  also  show  up  in  categories  1  and  2,  mirroring  the  performance  of  the 
gMTADS,  where  non-ordnance-related  items  were  generally  not  classed  as  likely  UXO, 
but  where  little  discrimination  capability  between  intact  UXO  and  UXO  fragments  was 
demonstrated.  That  is  not  surprising,  as  gMTADS  and  aMTADS  use  identical 
discrimination  procedures  and  algorithms. 

The  ORAGS  automatic  discrimination  algorithms  did  not  show  perfonnance  as 
good  as  the  manually  manipulated  dipole-fit  algorithms  used  for  the  MTADS  systems. 
Here,  9  intact  ordnance  items  were  category  1  picks,  10  were  category  2,  and  8  were 
category  3.  Categories  4-6  contained  a  total  of  10  intact  UXO,  and  17  intact  UXO  items 
were  not  detected  by  ORAGS.  Although  55  of  the  ordnance  fragment  items  were  not 
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detected  by  ORAGS,  those  that  were  generally  showed  up  in  categories  1-4  (94  of  103), 
again  showing  the  difficulty  that  magnetometer  discrimination  algorithms  appear  to  have 
in  sorting  intact  UXO  from  UXO  fragments. 
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Table  C-1.  Cross-Referenced  Data — ORAGS  then  aMTADS 


Intact  Ord 

Ord  Frag 

Nonord 

Geology 

No  Find 

gMTADS  Category  1  Totals 

54 

159 

7 

1 

0 

ORAGS 

aMTADS 

1 

1 

6 

11 

2 

3 

2 

3 

2 

4 

5 

6 

No  Match 

2 

1 

9 

16 

2 

1 

7 

3 

2 

4 

5 

6 

No  Match 

2 

3 

1 

8 

16 

2 

10 

3 

3 

4 

5 

2 

6 

No  Match 

4 

1 

4 

12 

2 

2 

6 

3 

3 

4 

1 

5 

6 

No  Match 

1 

1 

5 

1 

1 

3 

2 

2 

3 

2 

4 

5 

6 

No  Match 

2 

1 

6 

1 

2 

2 

1 

3 

1 

4 

5 

6 

No  Match 

1 

No  Match 

1 

11 

14 

2 

2 

3 

14 

3 

1 

12 

4 

1 

1 

5 

6 

No  Match 

1 

13 

2 
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Table  C-2.  Cross-Referenced  Data — aMTADS  then  ORAGS 


Intact  Ord 

Ord  Frag 

Nonord 

Geology 

No  Find 

gMTADS  Category  1  Totals 

54 
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7 

1 

0 
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ORAGS 

1 

1 

6 

11 

2 

9 

16 

3 

8 

16 

4 

4 

12 

5 

1 

3 

6 

2 

No  Match 

11 

14 

2 

2 

1 

3 

2 

2 

1 

7 

3 

10 

4 

2 

6 

5 

2 

6 

1 

No  Match 

3 

14 

3 

1 

2 

2 

2 

3 

3 

4 

3 

5 

2 

6 

1 

No  Match 

1 

12 

4 

1 

2 

3 

4 

1 

5 

6 

No  Match 

1 

1 

5 

1 

2 

3 

2 

4 

5 

6 

No  Match 

6 

1 

2 

3 

4 

5 

6 

No  Match 

No  Match 

1 

2 

2 

1 

3 

4 

1 

1 

5 

1 

1 

6 

1 

No  Match 

1 

13 

2 
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